Entertainment system, entertainment apparatus, recording medium, and program

ABSTRACT

An entertainment system for a game in which novel three-dimensional line drawing images are displayed on a display screen is disclosed. By pressing a predetermined control button of a manual controller at predetermined timing, a character object line drawing image can clear a virtual road object line drawing image having obstacle object line drawing images which move from the further right side of the screen toward the front left side of the screen.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation-in-part of U.S. patentapplication Ser. No. 09/687,577 filed on Oct. 13, 2000, which isassigned to the assignee of the present application.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an entertainment system, anentertainment apparatus, a recording medium, and a program fordisplaying a novel line drawing image according to music on a displayscreen to allow a game to be played.

2. Description of the Related Art

In some information apparatuses such as entertainment apparatusesincluding video game machines (entertainment systems), for example, agame is played by manipulating a controller while displaying thecontents of the game stored in a recording medium such as a CD-ROM onthe screen of a television receiver as a monitor.

Currently, many games available on the market are directed to utilizemore realistic and finer video images with the aid of recent advancedtechnology. In such games, the controller of the entertainment apparatuscan be vibrated according to the movement of images so as to make thegames more realistic and interesting. Under the circumstances, sincegames are getting more complicated, the difficulties of the games tendto be increased. In some games, high level of skills for manipulatingthe controller is required for the user. In this case, it is notpossible for some users such as amateur game players or older people tocomplete the games. Further, once a user completes such games andacquires the manipulation skills, the user may soon get tired of playingthe games.

In contrast, less complicated games utilizing only line drawing imagescan be widely accepted by people in different generations. That is,since such games are simple and do not require manipulation skills,children and old people can enjoy the heartwarming games.

SUMMARY OF THE INVENTION

The present invention was made taking the above-described points intoconsideration, and an object of the invention is to provide anentertainment system, an entertainment apparatus, a recording medium,and a program for displaying a novel line drawing image on a displayscreen to allow a game to be played.

Specifically, the object of the invention is to provide an entertainmentsystem, an entertainment apparatus, a recording medium, and a programfor displaying a novel line drawing image according to music on adisplay screen to allow a highly entertaining game to be played.

An entertainment system of the present invention comprises:

an entertainment apparatus for executing various programs;

a manual controller for inputting a manual control request of a user tothe entertainment apparatus; and

a display monitor for displaying an image outputted from theentertainment apparatus,

wherein the entertainment apparatus comprises:

means for analyzing an audio signal;

means for generating at least one substantially linear line drawingimage;

means for inserting a non-linear line drawing portion based on an resultof an analysis of the audio signal into the substantially linear linedrawing image to generate a modified substantially linear line drawingimage having the non-linear line drawing portion; and

means for generating a line drawing image of a character object;

wherein the line drawing image of the character object is displayed onthe modified substantially linear line drawing image having thenon-linear line drawing portion on the display monitor.

According to the entertainment system of the present invention, a linedrawing image of a character object is generated on a modifiedsubstantially linear line drawing image having a non-linear line drawingimage portion which has been inserted based on a result of analysis ofan audio signal. This makes it possible to display a novel line drawingimage according to music on the display.

In this case, two modified substantially linear line drawing images eachhaving a non-linear line drawing image portion are generated. Then, acharacter line drawing image is generated on each of the modifiedsubstantially linear line drawing images. Accordingly, it is possiblefor two users to play a match game (competition game).

In the above entertainment system according to the present invention,the entertainment apparatus may further comprise means for moving theline drawing image of the character object relative to the modifiedsubstantially linear line drawing image having the non-linear linedrawing portion. Accordingly, it is possible to generate a moreentertaining line drawing image.

Further, the entertainment apparatus may further comprise means forchanging the line drawing image of the character object into a linedrawing image of a different character object depending on how the linedrawing image of the character object moves on the modifiedsubstantially linear line drawing image having the non-linear linedrawing portion. Accordingly, it is possible to generate a still moreentertaining line drawing image.

Further, the entertainment apparatus may comprise means for impartingvibrations to the modified substantially linear line drawing imagehaving the non-linear line drawing portion and the line drawing image ofthe character object. Accordingly, it is possible to generate a quiteentertaining line drawing image.

In this case, each of the line drawing images may be drawn as athree-dimensional line drawing image to generate a highly entertainingimage which is less likely to become tiresome.

Further, an audio signal may be used which is supplied to theentertainment apparatus from a recording medium or which is downloadedthereto via communication.

An entertainment apparatus of the present invention is connectable to amanual controller for inputting a manual control request of a user tothe entertainment apparatus, and connectable to a display monitor fordisplaying an image outputted from the entertainment apparatus,

wherein the entertainment apparatus comprises:

means for analyzing an audio signal;

means for generating at least one substantially linear line drawingimage;

means for inserting a non-linear line drawing portion based on a resultof an analysis of the audio signal into the substantially linear linedrawing image to generate a modified substantially linear line drawingimage having the non-linear line drawing portion; and

means for generating a line drawing image of a character object;

wherein the line drawing image of the character object is displayed onthe modified substantially linear line drawing image having thenon-linear line drawing portion on the display monitor.

According to the entertainment apparatus of the present invention, aline drawing image of a character object is generated on a modifiedsubstantially linear line drawing image having a non-linear line drawingimage portion which has been inserted based on a result of analysis ofan audio signal. This makes it possible to display a novel line drawingimage according to music on the display.

In this case, two modified substantially linear line drawing images eachhaving a non-linear line drawing image portion are generated. Then, acharacter line drawing image is generated on each of the modifiedsubstantially linear line drawing images. Accordingly, it is possiblefor two users to play a match game (competition game).

A recording medium of the present invention comprises the steps of:

analyzing an audio signal;

generating at least one substantially linear line drawing image;

inserting a non-linear line drawing portion based on a result of ananalysis of the audio signal into the substantially linear line drawingimage to generate a modified substantially linear line drawing imagehaving the non-linear line drawing portion; and

generating a line drawing image of a character object on the modifiedsubstantially linear line drawing image having the non-linear linedrawing portion.

According to the recording medium of the present invention, a linedrawing image of a character object is generated on a modifiedsubstantially linear line drawing image having a non-linear line drawingimage portion which has been inserted based on a result of analysis ofan audio signal. This makes it possible to display a novel line drawingimage according to music on the display.

In this case, two modified substantially linear line drawing images eachhaving a non-linear line drawing image portion are generated. Then, acharacter line drawing image is generated on each of the modifiedsubstantially linear line drawing images. Accordingly, it is possiblefor two users to play a match game (competition game).

The recording medium is applicable to a recording medium for recordingprograms and data used for an entertainment system comprising:

an entertainment apparatus for executing various programs;

a manual controller for inputting a manual control request of a user tothe entertainment apparatus; and

a display monitor for displaying an image outputted from theentertainment apparatus.

In the above recording medium of the present invention, the program mayfurther comprise the step of moving the line drawing image of thecharacter object relative to the modified substantially linear linedrawing image having the non-linear line drawing portion. Accordingly,it is possible to generate a more entertaining line drawing image.

Further, the program may further comprise the step of changing the linedrawing image of the character object into a line drawing image of adifferent character object depending on how the line drawing image ofthe character object moves on the modified substantially linear linedrawing image having the non-linear line drawing portion. Accordingly,it is possible to generate a still more entertaining line drawing image.

Further, the program may further comprise the step of impartingvibrations to the modified substantially linear line drawing imagehaving the non-linear line drawing image portion and the line drawingimage of the character object. Accordingly, it is possible to generate aquite entertaining line drawing image.

In this case, each of the line drawing images may be drawn as athree-dimensional line drawing image to generate a highly entertainingimage which is less likely to become tiresome.

A program of the present invention comprises the steps of:

analyzing an audio signal;

generating at least one substantially linear line drawing image;

inserting a non-linear line drawing portion based on a result of ananalysis of the audio signal into the substantially linear line drawingimage to generate a modified substantially linear line drawing imagehaving the non-linear line drawing portion; and

generating a line drawing image of a character object on the modifiedsubstantially linear line drawing image having the non-linear linedrawing portion.

According to the program of the present invention, a line drawing imageof a character object is generated on a modified substantially linearline drawing image having a non-linear line drawing image portion whichhas been inserted based on a result of analysis of an audio signal. Thismakes it possible to display a novel line drawing image according tomusic on the display.

In this case, two modified substantially linear line drawing images eachhaving a non-linear line drawing image portion are generated. Then, acharacter line drawing image is generated on each of the modifiedsubstantially linear line drawing images. Accordingly, it is possiblefor two users to play a match game (competition game).

The program is applicable to a program for use of an entertainmentsystem comprising:

an entertainment apparatus for executing various programs;

a manual controller for inputting a manual control request of a user tothe entertainment apparatus; and

a display monitor for displaying an image outputted from theentertainment apparatus.

The above and other objects, features and advantages of the presentinvention will become more apparent from the following description whentaken in conjunction with the accompanying drawings in which a preferredembodiment of the invention is shown by way of illustrative example.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an entertainment system according to anembodiment of the present invention.

FIG. 2 is a perspective view of a manual controller.

FIG. 3 is a block diagram showing a circuit configuration of theentertainment system.

FIG. 4 is a block diagram showing a circuit configuration of the manualcontroller.

FIG. 5 is a flow chart for explaining the operation of the entertainmentsystem as a whole.

FIG. 6 is an illustration of a game starting screen.

FIG. 7 is an illustration of a name registration screen.

FIG. 8 is an illustration of the name registration screen.

FIG. 9 is an illustration of a game selection screen.

FIG. 10 is an illustration of the game selection screen.

FIG. 11 is a flow chart showing details of game processing.

FIG. 12 is an illustration of character objects.

FIG. 13 is an illustration of obstacle objects.

FIG. 14 is an illustration of a virtual road object.

FIG. 15 is an illustration of an example of formation of an object.

FIG. 16 is an illustration of a table showing correspondence betweencontrol buttons and obstacle objects.

FIG. 17 is a flow chart for explaining an audio signal analyzingprocess.

FIG. 18 shows a table of correspondence between results of audio signalanalysis and obstacle objects to be generated.

FIG. 19 shows a table of correspondence between results of audio signalanalysis and obstacle objects to be generated.

FIG. 20 is a flow chart for explaining a line drawing display updatingprocess.

FIG. 21 is an illustration of a frame buffer.

FIG. 22 is an illustration for explaining generation of athree-dimensional line drawing image.

FIG. 23 is an illustration of a screen that appears immediately afterthe beginning of a game.

FIG. 24 is an illustration for explaining a vibration process.

FIG. 25 is an illustration showing a screen that appears several secondsafter the beginning of the game.

FIG. 26 is an illustration of a screen in which a character object getsover an obstacle object.

FIG. 27 is an illustration of a screen in which the character objectrolls over an obstacle object.

FIG. 28 is an illustration of a screen in which the character objectstrides over an obstacle object.

FIG. 29 is an illustration of a screen in which the character objectrolls in an obstacle object.

FIG. 30 is an illustration of a screen which appears immediately afterthe character object fails in getting over an obstacle object.

FIG. 31 is an illustration of a screen which appears at a time intervalof about one second or more after the character object fails in gettingover the obstacle object.

FIG. 32 shows a table of character status.

FIG. 33 is an illustration of a Game Over screen.

FIG. 34 is an illustration of the Game Over screen.

FIG. 35 is an illustration of an Ending screen.

FIG. 36 is a block diagram showing an image processing/audio processingfunction.

FIG. 37 is an illustration for explaining an example of a screenaccording to an embodiment of a game for two players.

FIG. 38 is an illustration for explaining another example of a screenaccording to the embodiment of a game for two players.

FIG. 39 is an illustration for explaining still another example of ascreen according to the embodiment of a game for two players.

FIG. 40 shows an example of a table of correspondence between controlbuttons and obstacle objects according to another embodiment of theinvention.

FIG. 41 is an illustration for explaining creation of an obstacle objectaccording to the embodiment.

FIG. 42 is an illustration for explaining obstacle objects according tothe embodiment.

FIG. 43 is an illustration of a screen in which obstacle objects rotateabout a virtual road.

FIG. 44 is a view showing a waveform of a digital audio signal.

FIG. 45 is a view showing distinctive points in the waveform of thedigital audio signal.

FIG. 46 is a view showing a waveform of an emphasized signal generatedby emphasizing the digital audio signal in a predetermined process.

FIG. 47 is a view showing a waveform of a signal (attack events)generated by converting the emphasized signal with a threshold toeliminate unnecessary parts of the waveform.

FIG. 48 is a view showing peaks of the respective attack events(potential events) in the waveform.

FIG. 49 is a view showing final events selected from the potentialevents in the waveform by a predetermined process.

FIG. 50 is a view showing positions of the final events in the waveformof the digital audio signal.

FIG. 51 is a view partially showing the waveform in FIG. 41 which isenlarged on the time axis.

FIG. 52 is a view illustrating a power of an audio event at a certaintime point.

FIG. 53 is a graph showing short term powers.

FIG. 54 is a graph showing the short term powers and long term powers.

FIG. 55 is a view showing the waveform of the emphasized signal as theration of the short term power to the long term power.

FIG. 56 is a view showing attack events in the waveform which is dividedinto select periods.

FIG. 57 is a view showing potential events representing peaks inrespective select periods of the waveform.

FIG. 58 is a view showing the waveform of potential events in whichshadow periods are set on the time axis of the waveform.

FIG. 59 is a view showing final events in the waveform.

DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention will be described belowspecifically with reference to drawings.

FIG. 1 shows generally an arrangement of an entertainment system 10 towhich an image processing apparatus according to the embodiment of thepresent invention is applied.

The entertainment system 10 basically comprises an entertainmentapparatus 12 for executing various programs, a memory card 14 detachablyconnected to the entertainment apparatus 12, a manual controller 16detachably connected to the entertainment apparatus 12 by a connector62, and a monitor 18 such as a television receiver which is suppliedwith video and audio output signals from the entertainment apparatus 12.

The entertainment apparatus 12 reads a program and data recorded in amass storage medium such as an optical disk 20 such as a CD-ROM or thelike, and executes a game, for example, based on the program dependingon commands supplied from a user, e.g., a game player, via the manualcontroller 16. The execution of the game mainly represents controllingthe progress of the game by controlling the display of images and thegeneration of sounds on the monitor 18 based on manual input actionsentered from the manual controller 16 via the connector 62.

The entertainment system 12 is capable of playing back an optical disk20 such as a CD (compact disk) as a recording medium. Specifically,audio signals as music data (sound data) are read and played back byreferring TOC (table of contents) data stored in the compact disk.

Further, the entertainment apparatus 12 is capable of executing a gameprogram by utilizing the TOC data and music data stored in the compactdisk.

The recording medium for supplying the application program and sounddata is not limited to the optical disk 20. Alternatively, theentertainment apparatus 12 may be supplied with the application programand sound data via a communication link, rather than being supplied fromthe optical disk 20 as the recording medium.

The entertainment apparatus 12 has a substantially flat casing in theshape of a rectangular parallelepiped which houses a disk loading unit22 disposed centrally for loading the optical disk 20 for supplying theapplication program and data for a video game or the like. The casingsupports a reset switch 24 for resetting a program which is beingpresently executed, a disk control switch 26 for controlling the loadingof the optical disk 20, a power supply switch 28, and two slots 30, 32.

The slots 30, 32 have respective upper slot units 30B, 32B andrespective lower slot units 30A, 32A. Two manual controllers 16 may beconnected respectively to the lower slot units 30A, 32A via theconnectors 62, and memory cards 14 for storing flags indicative ofinterim game data may be connected respectively to the upper slot units30B, 32B.

As shown in FIG. 1, when two manual controllers 16 connected to the slotunits 30A, 32A via the connectors 62, two users can play a match game(competition game).

The match game (competition game) may be played by more than two gameplayers. In this case, a conventional multi-tap adapter (not shown) isconnected to the slot unit 30A. The multi-tap adapter is provided withmore than two slots having the shape of the slot unit 30A. Then, manualcontrollers 16 are connected to the slot units of the multi-tap adapter.In this manner, more than two manual controllers 16 can be connected tothe entertainment apparatus 12 via the multi-tap adapter.

The slots 30, 32 (the upper slot units 30B, 32B and the lower slot units30A, 32A) are asymmetrically shaped to prevent the connectors 62 and thememory cards 14 from being inserted in the wrong direction.

As shown in FIGS. 1 and 2, the manual controller 16 basically comprisesfirst and second control pads 34, 36, an L (Left) button 38L, an R(Right) button 38R, a start button 40, and a selection button 42. Themanual controller 16 also has joysticks 44, 46 for making analog controlactions, a mode selection switch 48 for selecting control modes of thejoysticks 44, 46, and an indicator 50 for indicating a selected controlmode. The indicator 50 comprises a light-emitting element such as alight-emitting diode or the like.

As shown in FIG. 2, the manual controller 16 has a housing 104comprising an upper member 100 and a lower member 102 which are matedand joined to each other by fasteners such as screws.

As shown in FIG. 2, a pair of left and right grips 106, 108 projectsfrom one side of respective opposite ends of the housing 104. The leftand right grips 106, 108 are shaped so as to be gripped by the palms ofleft and right hands of the user or game player when the manualcontroller 16 is connected to the entertainment apparatus 12 andinformation retrieval is carried out or the game is played thereby, forexample.

As shown in FIG. 1, the left and right grips 106, 108 are progressivelyspaced away from each other toward their distal ends.

As shown in FIGS. 2, the first control pad 34 is disposed on one end ofthe housing 104 and comprises a first pressable control member (upbutton) 110 a, a second pressable control member (right button) 110 b, athird pressable control member (down button) 110 c, and a fourthpressable control member (right button) 110 d. The first through fourthpressable control members 110 a, 110 b, 110 c, 110 d project on an uppersurface of the housing 104 and are arranged in a crisscross pattern.

The first control pad 34 includes switch elements as signal inputelements associated respectively with the first through fourth pressablecontrol members 110 a, 110 b, 110 c, 110 d. The first control pad 34functions as a directional controller for controlling the direction ofmovement of a displayed game character, for example. When the gameplayer selectively presses the first through fourth pressable controlmembers 110 a, 110 b, 110 c, 110 d to turn on or off the switch elementsassociated respectively with the first through fourth pressable controlmembers 110 a, 110 b, 110 c, 110 d, the displayed game character movesin the direction corresponding to the pressed one of the first throughfourth pressable control members 110 a, 110 b, 110 c, 110 d.

As shown in FIGS. 1 and 2, the second control pad 36 is disposed on theother end of the housing 104 and comprises a first pressable controlmember (Δ button) 112 a, a second pressable control member (◯ button)112 b, a third pressable control member (X button) 112 c, and a fourthpressable control member (□ button) 112 d. The first through fourthpressable control members 112 a, 112 b, 112 c, 112 d project on theupper surface of the housing 104 and are arranged in a crisscrosspattern.

The first through fourth pressable control members 112 a, 112 b, 112 c,112 d are constructed as independent members, and associated withrespective switch elements disposed in the second control pad 36.

The second control pad 36 serves as a function setting/performing unitfor setting functions for a displayed game character assigned to thepressable control members 112 a-112 d or performing functions of adisplayed game character when the switch elements associated with thepressable control members 112 a-112 d are turned on.

The L button 38L and the R button 38R are disposed on a side of thehousing 104 remote from the first and second grips 106, 108 andpositioned respectively at the opposite ends of the housing 104. Asshown in FIG. 2, the L button 38L and the R button 38R have respectivefirst and second pressable control members 114 a, 114 b and 116 a, 116 band respective switch elements associated respectively with thepressable control members 114 a, 114 b and 116 a, 116 b.

The L button 38L and the R button 38R serve as respective functionsetting/performing units for setting functions for a displayed gamecharacter assigned to the pressable control members 114 a, 114 b and 116a, 116 b or performing functions of a displayed game character when theswitch elements associated with the pressable control members 114 a, 114b and 116 a, 116 b are turned on.

The first pressable control members 114 a, 114 b are also referred to asthe L1 button 114 a, the L2 button 114 b, respectively. The secondpressable control members 116 a, 116 b are also referred to as the R1button 116 a, the R2 button 114 b, respectively.

As shown in FIG. 2, the manual controller 16 also has left and rightanalog control pads 118, 120 disposed respectively at confrontingcorners defined between the housing 104 and the proximal ends of thefirst and second grips 106, 108 which are joined to the housing 104.

The left and right analog control pads 118, 120 have the respectivejoysticks 44, 46 which can be tilted in all directions 360° aboutcontrol shafts thereof, and respective signal input elements such asvariable resistors or the like which are operable by the respectivejoysticks 44, 46. Specifically, the joysticks 44, 46 are mounted on tipends of the control shafts that are normally urged to return to theirneutral positions by resilient members, and can be tilted in alldirections (360°) about the axes of the control shafts.

The left and right analog control pads 118, 120 can move a displayedgame character while rotating the same or while changing its speed, andcan make an analog-like action such as to change the form of a displayedcharacter, when the game player rotates the joysticks 44, 46. Therefore,the left and right analog control pads 118, 120 are used as a controlunit for entering command signals for a displayed character to performthe above movement or action.

When the mode selection switch 48 is pressed, it can select a controlmode for allowing a command signal to be inputted from the left andright analog control pads 118, 120 or a control mode for inhibiting acommand signal from being inputted from the left and right analogcontrol pads 118, 120.

When the mode selection switch 48 is pressed, it can also select acontrol mode for allowing a command signal to be inputted from the leftand right analog control pads 118, 120 and selecting the function of thefirst through fourth pressable control members 112 a, 112 b, 112 c, 112d of the second control pad 36 or the function of the pressable controlmembers 114 a, 114 b and 116 a, 116 b of the L button 38L and the Rbutton 38R. Depending on the control mode selected by the mode selectionswitch 48, the mode indicator 50 flickers and changes its indicationlight.

As shown in FIG. 2, the first and second grips 106, 108 projecting fromthe housing 104 are gripped respectively by the palms of the hands ofthe game player. The housing 104 is not required to be supported byfingers, and the manual controller 16 can be held by the hands while atleast six out of the ten fingers of the hands can freely be moved.

As shown in FIG. 2, when the first and second grips 106, 108 are grippedrespectively by the palms of the hands of the game player, the thumbsRf1, Lf1 of the right and left hands can extend over the joysticks 44,46 of the left and right analog control pads 118, 120, the first throughfourth pressable control members 110 a-110 d of the first control pad34, and the first through fourth pressable control members 112 a-112 dof the second control pad 36, and can selectively press the joysticks44, 46, the pressable control members 110 a-110 d, and the pressablecontrol members 112 a-112 d.

Since the joysticks 44, 46 of the left and right analog control pads118, 120 are positioned in confronting relation to the proximal ends ofthe first and second grips 106, 108 which are joined to the housing 104,when the first and second grips 106, 108 are gripped by the left andright hands, the joysticks 44, 46 are positioned most closely to thethumbs Rf1, Lf1, respectively. Therefore, the joysticks 44, 46 caneasily be rotated by the thumbs Rf1, Lf1.

As shown in FIG. 2, when the first and second grips 106, 108 are grippedrespectively by the palms of the hands of the game player, the indexfingers Rf2, Lf2 and middle fingers Rf3, Lf3 of the right and left handscan extend over positions where they can selectively press the first andsecond pressable control members 114 a, 114 b and 116 a, 116 b of the Rbutton 38R and the L button 38L.

Further, the manual controller 16 is provided with unillustratedvibration imparting mechanisms comprising motors or the like forimparting vibrations to the user in order for the user to be able toplay a highly realistic game. Vibration commands for energizing thevibration imparting mechanisms are generated by the entertainmentapparatus 12 so as to produce suitable vibration effects in the game.

Next, circuit arrangements of the entertainment apparatus 12 and themanual controller 16 will be described below.

FIG. 3 shows an arrangement of the entertainment system 10 including acircuit arrangement of major electric components of the entertainmentapparatus 12.

As shown in FIG. 3, the entertainment apparatus 12 comprises a controlsystem 250 including a central processing unit (CPU) 251 and itsperipheral devices, a graphic system 260 including a graphic processingunit (GPU) 262 for generating and storing image data in a frame buffer263, a sound system 270 including a sound processing unit (SPU) 271 forgenerating music sounds and sound effects, an optical disk controller280 for controlling an optical disk 20 in which application programs arerecorded, a communication controller 290 for controlling signals fromthe manual controller 16 which enter instructions from the user, anddata supplied to and from a memory card 14 which stores game settings,and a bus BUS to which the control system 250, the graphic system 260,the sound system 270, the optical disk controller 280, and thecommunication controller 290 are connected.

The control system 250 comprises a CPU 251, a peripheral devicecontroller 252 for controlling interrupts and direct memory access (DMA)data transfer, a main memory 253 comprising a random-access memory(RAM), and a read-only memory (ROM) 254 which stores various programssuch as an operating system for managing the main memory 253, thegraphic system 260, the sound system 270, etc. The main memory 253 is amemory capable of storing a program which is being executed.

The CPU 251 controls the entertainment apparatus 12 in its entirety byexecuting the operating system stored in the ROM 254. The CPU 251comprises a 32-bit RISC-CPU, for example.

When the entertainment apparatus 12 is turned on, the CPU 251 executesthe operating system stored in the ROM 254 to start controlling thegraphic system 260, the sound system 270, etc. For example, when theoperating system is executed, the CPU 251 initializes the entertainmentapparatus 12 in its entirety for checking its operation, and thereaftercontrols the optical disk controller 280 to execute an applicationprogram recorded in the optical disk 20 loaded in the disk loading unit22 (see FIG. 1)

As the application program such as a game program stored in the opticaldisk 20 is executed, the CPU 251 controls the graphic system 260, thesound system 270, etc. depending on commands entered from the user forthereby controlling the display of images and the generation of musicsounds and sound effects.

The graphic system 260 comprises a geometry transfer engine (GTE) 261for performing coordinate transformations including perspectivetransformations and other processing, a GPU 262 for generating imagedata according to instructions from the CPU 251, a frame buffer 263 forstoring image data generated by the GPU 262 and updating a screen imageeach time a screen switching signal (screen image switching signal) suchas a vertical synchronization signal is generated, and an image decoder264 for decoding image data compressed and encoded by an orthogonaltransform such as a discrete cosine transform. The image data stored inthe frame buffer 263 is outputted by means of GPU 262 as a video imagedata. The outputted video image data is supplied to a display 18A of themonitor 18 such as television receiver or the like via an outputterminal. The image data (including three dimensional image data) isupdated each time a vertical synchronization signal is generated.

The GTE 261 has a parallel arithmetic mechanism for performing aplurality of arithmetic operations parallel to each other, and canperform coordinate transformations (including perspectivetransformations for transforming three dimensional images into twodimensional images), light source calculations, matrixes, or vectors ata high speed in response to a request from the CPU 251. Specifically,the GTE 261 can calculate the coordinates of a maximum of 1.5 millionpolygons per second for a flat shading process to plot one triangularpolygon with one color, for example. With the GTE 261, the entertainmentapparatus 12 is able to reduce the burden on the CPU 351 and performhigh-speed coordinate calculations.

According to an image generating instruction from the CPU 251, the GPU262 generates and stores the data of a polygon or the like in the framebuffer 263. The GPU 262 is capable of generating and storing a maximumof 360 thousand polygons per second.

The frame buffer 263 comprises a dual-port RAM, and is capable ofsimultaneously storing image data generated by the GPU 262 or image datatransferred from the main memory 53, and reading image data for display.

The frame buffer 263 has a storage capacity of 1 Mbytes, for example,and is handled as a 16-bit matrix made up of a horizontal row of 1024pixels and a vertical column of 512 pixels. The frame buffer 263 hasareas for selectively storing image data and outputting the stored imagedata as video output data, a CLUT (color look-up table) area for storinga color look-up table which will be referred to by the GPU 262 when itgenerates a polygon or the like, and a texture area for storing texturedata to be subjected to coordinate transformations when a polygon isgenerated and mapped onto a polygon generated by the GPU 262. The CLUTarea and the texture area are dynamically varied as the areas forselectively storing image data and outputting the stored image data asvideo output data are varied.

The GPU 262 can perform, in addition to the flat shading process, aGouraud shading process for determining colors in polygons byinterpolating intensities from the vertices of the polygons, and atexture mapping process for mapping textures stored in the texture areasonto polygons. For performing the Gouraud shading process or texturemapping process, the GTE 261 can perform coordinate calculations for amaximum of about 500,000 polygons per second.

The image decoder 264 is controlled by the CPU 251 to decode image dataof a still or moving image stored in the main memory 253, and store thedecoded image into the main memory 253.

Image data reproduced by the image decoder 264 is transferred to theframe buffer 263 by the GPU 262, and can be used as a background for animage plotted by the GPU 262.

The sound system 270 comprises an SPU 271 for generating music sounds,sound effects, etc. based on instructions from the CPU 251, a soundbuffer 272 for storing waveform data from the SPU 271. Music sounds,sound effects generated by the SPU 271 are outputted by a speaker 18B ofthe monitor 18.

The SPU 271 has an ADPCM (adaptive differential PCM) function forreproducing 16-bit sound data which has been encoded as 4-bitdifferential sound data by ADPCM, a reproducing function for reproducingthe waveform data stored in the sound buffer 272 to generate soundeffects, etc., and a modulating function for modulating and reproducingthe waveform data stored in the sound buffer 272.

The sound system 270 can be used as a sampling sound source whichgenerates music sounds, sound effects, etc. based on the waveform datastored in the sound buffer 272 according to commands from the CPU 251.

The optical disk controller 280 comprises an optical disk drive 281 forreproducing application programs and data recorded on the optical disk20, a decoder 282 for decoding programs and data that are recorded withan error correcting code (ECC) added thereto, and a buffer 283 fortemporarily storing data read from the optical disk drive 281 so as toallow the data from the optical disk 20 to be read at a high speed. Anauxiliary CPU 284 is connected to the decoder 282.

Sound data recorded on the optical disk 20 which is read by the opticaldisk drive 281 includes PCM data converted from analog sound signals, inaddition to the ADPCM data. The ADPCM data, which is recorded as 4-bitdifferential data of 16-bit digital data, is decoded by the decoder 82,supplied to the SPU 271, converted thereby into analog data, and appliedto drive the speaker 18B. The PCM data, which is recorded as 16-bitdigital data, is decoded by the decoder 282 and then applied to drivethe speaker 18B.

The communication controller 290 comprises a communication controller291 for controlling communication with the CPU 251 via the bus BUS. Thecommunication controller 291 is connected to the manual controller 16for entering commands from the user, the memory card 14 as an auxiliarymemory device for storing game settings, etc. and an unillustratedportable electronic device.

As shown in FIGS. 1 and 2, the manual controller 16 has more than 10command keys for entering commands from the user, and transmits statusesof the command keys about 60 times per second to the communicationcontroller 291 by way of synchronous communication according to aninstruction from the communication controller 291. The communicationcontroller 291 transmits the statuses of the command keys to the CPU251.

In this manner, commands from the user are applied to the CPU 251, whichcarries out a process according to the commands based on the gameprogram being executed.

A large amount of image data needs to be transferred at high speedbetween the main memory 253, the GPU 262, the image decoder 264, and thedecoder 282 for reading a program, displaying an image, or generatingand storing image data.

In the entertainment apparatus 12, data is transferred directly betweenthe main memory 253, the GPU 262, the image decoder 264, and the decoder282 according to the DMA data transfer under the control of theperipheral device controller 252, rather than the CPU 251. Therefore,the burden on the CPU 251 can be reduced for data transfer, andhigh-speed data transfer can be achieved between the main memory 253,the GPU 262, the image decoder 264, and the decoder 282.

When setting data of a game being executed need to be stored, the CPU251 transmits the setting data to the communication controller 291,which writes the transmitted setting data into the memory card 14 or theunillustrated portable electronic device which is inserted in the slot30B, 32B.

The memory card 14 is provided with a main body interface for connectionto the entertainment apparatus 12, and a memory interface for outputtingdata to and inputting data from a nonvolatile memory incorporatedtherein.

The communication controller 291 (see FIG. 3) has a built-in protectioncircuit for protection against electric breakdown. The memory card 10and the portable terminal 100 are separate from the bus BUS, and can beconnected and disconnected while the entertainment apparatus 12 is beingenergized. Therefore, when the memory card 14 suffers a storage capacityshortage, a new memory card can be connected without having to turn offthe entertainment apparatus 12. Consequently, any game data that need tobe backed up can be stored in a new memory card 14 connected to theentertainment apparatus 12, without the danger of being lost.

As shown in FIG. 3, the entertainment apparatus 12 further includes aparallel I/O interface (PIO) 296 and a serial I/O interface (SIO) 297which serve to connect external extended devices to the entertainmentapparatus 12. For example, the parallel I/O interface 296 can beconnected to a compact disk player or a DAT (digital audio taperecorder) for playing back music data. The operations (power ON/OFF,music reproduction, stop, skip, and music selection) of the compact diskplayer and DAT can be controlled by the CPU 251. The serial I/Ointerface 297 can be connected to a personal digital assistant such asthe unillustrated portable electronic device.

The entertainment apparatus 12 is capable of executing a program storedin the optical disk 20 by means of the optical disk drive 281, whilereading digital audio signals from a music player 298 via the PIO 296simultaneously.

As shown in FIG. 4, the bidirectional communication function between theentertainment apparatus 12 and the manual controller 16 can be performedwhen the connector 62 capable of performing bidirectional serialcommunications with the manual controller 16 is connected to theentertainment apparatus 12.

A system in the manual controller 16 for performing the bidirectionalcommunication function comprises a serial I/O interface SIO forperforming serial communication with the entertainment apparatus 12, aparallel I/O interface PIO for entering control data from a plurality ofcontrol buttons, a one-chip microcomputer comprising a CPU, a RAM, and aROM, and a motor driver 150 for energizing the motors 130 of thevibration imparting mechanisms. Each of the motors 130 is energized forrotation by a voltage and a current supplied from the motor driver 150.

As described above, the manual controller 16 has more than 10 controlbuttons PB such as the up button 110 a, the right button 110 b, the leftbutton 110 c, the down button 110 d, the Δ button 112 a, the ◯ button112 b, the X button 112 c, the □ button 112 d, the L1 button 114 a, theL2 button 114 b, the R1 button 116 a, the R2 button 116 b.

A system in the entertainment apparatus 12 for performing thebidirectional communication function comprises a serial I/O interfaceSIO for performing serial communication with the manual controller 16.When the connector 62 is connected to the serial I/O interface SIO ofthe entertainment apparatus 12, the serial I/O interface SIO of theentertainment apparatus 12 is connected to the serial I/O interface SIOof the manual controller 16 via the connector 62 for performingbidirectional communications between the manual controller 16 and theentertainment apparatus 12. Other structural details of theentertainment apparatus 12 are omitted from illustration in FIG. 4.

Signal and control lines for bidirectional serial communications includea data transfer signal line TXD (Transmit X′ for Data) for sending datafrom the entertainment apparatus 12 to the manual controller 16, a datatransfer signal line RXD (Received X′ for Data) for sending data fromthe manual controller 16 to the entertainment apparatus 12, a serialsynchronous clock signal line SCK (Ser. Clock) for extracting data fromthe data transfer signal lines TXD, RXD, a control line DTR (DataTerminal Ready) for establishing and cutting off communication with themanual controller 16 as a terminal, and a flow control line DSR (DataSet Ready) for transferring a large amount of data.

The signal and control lines for bidirectional serial communication areaccommodated in a cable. As shown in FIG. 4, this cable further includesa power line 152 extending from a power supply in the entertainmentapparatus 12 and connected to the motor drivers 150 in the manualcontroller 16 for supplying electric energy to energize the motors 130and other components of the manual controller 16.

A process of bidirectional serial communication between the manualcontroller 16 and the entertainment apparatus 12 will be describedbelow. In order for the entertainment apparatus 12 to communicate withthe manual controller 16 to read control data of the control buttons(button information) of the first and second control pads 34, 36 and theL button 38L and the R button 38R, the entertainment apparatus 12 firstoutputs selection data to the control line DTR. As a result, the manualcontroller 16 confirms that it is selected by the control line DTR, andthen waits for a signal from the signal line TXD. Then, theentertainment apparatus 12 outputs an identification code indicative ofthe manual controller 16 to the data transfer signal line TXD. Themanual controller 16 receives the identification code from the signalline TXD.

When the manual controller 16 recognizes the identification code, themanual controller 16 starts communicating with the entertainmentapparatus 12. The entertainment apparatus 12 sends control data via thedata transfer signal line TXD to the manual controller 16, which sendscontrol data produced by a control button via the data transfer signalline RXD to the entertainment apparatus 12. In this manner, theentertainment apparatus 12 and the manual controller 16 performbidirectional serial communications. The bidirectional serialcommunications will be finished when the entertainment apparatus 12outputs selection stop data via the control line DTR.

With the bidirectional serial communication function, the manualcontroller 16 can send mainly control data of control buttons PB to theentertainment apparatus 12, and the entertainment apparatus 12 can senda vibration generating command for energizing the motors 130 of thevibration imparting mechanisms 128 via the data transfer signal line TXDto the manual controller 16.

The vibration generating command for energizing the motors 130 isestablished in advance in a CD-ROM set in the entertainment apparatus12.

A description will be made with reference to the flow chart shown inFIG. 5 on functions and operations characteristic of the entertainmentsystem 10 of the present embodiment.

First, the monitor 18, memory card 14 and manual controller 16 areconnected to the entertainment apparatus 12. Further, the optical disk20 is loaded in the disk loading unit 22. The optical disk 20 is arecording medium such as a CD-ROM in which various functions arerecorded as programs and data.

In this state, when the power supply switch 28 is pressed at step S1,power is supplied to the entertainment apparatus 12 from an AC powersource (not shown).

When power is supplied, the CPU 251 starts operating on the operatingsystem stored in the ROM 254 at step S2 to perform initialization suchas writing of required programs and data (including initial screen dataand initial music data) read from the ROM 254 in the main memory 253.

At step S3, the initial screen data is drawn in the frame buffer 263through the image decoder 264 and the GPU 262 under the control of theperipheral device controller 252, and the drawn initial screen data issupplied through the GPU 262 to the display 18A of the monitor 18 asvideo output to display an initial screen on the display 18A. At thistime, the initial music data stored in the ROM 254 is supplied to thesound buffer 272 through the SPU 271, and the stored initial screen datais supplied through the SPU 271 to the speaker 18B of the monitor 18 asaudio output to generate music (pieces of music) from the speaker 18B insynchronism with the initial screen.

Next, at step S4, the state of the decoder 282 is checked by, forexample, the CPU 251 to confirm the presence of the optical disk 20 inthe disk loading unit 22 by checking whether writing of programs anddata read from the optical disk drive 281 in the buffer 283 through thedecoder 282 has occurred as a result of automatic activation caused byloading of the optical disk 20 which is a CD-ROM.

Actually, while the optical disk 20 is not being loaded in the diskloading unit 22, the display of the initial screen at step S3 continues.When the optical disk 20 is loaded into the disk loading unit 22, theprocess proceeds to the next step S5.

At the process of step S5, the programs and data read from the opticaldisk 20 are directly stored in the memory 253 through the decoder 282under the control of the auxiliary CPU 284 or stored in the main memory253 through the buffer 283.

In the following description, images are processed by CPU 251 or GPU262.

FIG. 6 shows a start screen 300 displayed on the display 18A at theprocess of step S5.

In the start screen 300, vibrating images of an alphabetical expression“Vibribbon”, English words “Push Start”, and several asterisks or thelike are displayed. Each of these images is a three-dimensional linedrawing image having a predetermined length or a three-dimensional imagewhich is separated into parts having predetermined lengths. In thisstate, for example, the expression “Vibribbon” rotates along acircumferential wall of a virtual transparent column about the axisthereof in the lateral direction of the screen at a predetermined timeinterval such that the expression “Vibribbon” integrally moves to thefurther side of the screen and then returns to the front side of thescreen.

A detailed description will be made later on a process of generating athree-dimensional vibrating line drawing image having a predeterminedlength or a three-dimensional vibrating line drawing image which isseparated into parts having predetermined lengths, the process being afundamental feature of the display process according to the invention(the process is also referred to as “a three-dimensional line drawingimage irregular display process”.

When it is determined at step S6 that the start button 40 of the manualcontroller 16 has been pressed with the start screen 300 displayed asshown in FIG. 6, a process of registering the name of the user (gameplayer) is performed at step S7.

At the step S7, a name registration process screen 302 as shown in FIG.7 is displayed on the display 18A. On the name registration processscreen 302, a presently selected region (see the alphabet “S” in FIG. 7)is enlarged, and each character or symbol is displayed using irregulardisplay of three-dimensional line drawing images. The name of the user,e.g., “POKEPOKE” is then alphabetically input by manipulating thecontrol buttons PB of the manual controller 16. The control buttons PBare manipulated to move a cursor to the position of “OK” as shown on adisplay screen 304 in FIG. 8 (the cursor is displayed in a positionwhich is enlarged and displayed using irregular display ofthree-dimensional line drawing images), and the “◯” button 112 b ispressed to store (register) the input name in the main memory 253.

At step S8, as shown in FIG. 9, a game selection screen 306 for a gameselecting process is displayed on the display 18A. On this screen, athree-dimensional line drawing image is displayed. The three-dimensionalline image comprises a decagonal object 308 and names of the selectabletypes of games or the like positioned on straight lines extendingoutwardly from vertices of the decagonal object 308. In this “vibribbongame” (“vibribbon” means a vibrating ribbon), three types (levels) ofgames at different difficulties such as “easy”, “normal” and “hard” areavailable, for example. The type of the presently selected game is“easy”. In the vibribbon game, a game character is controlled accordingto music stored in advance in the optical disk 20. Specifically, twopieces of music are selectably recorded in the optical disk 20 for eachof the three types of games.

Further, when an “endless” mode is selected on the game selection screen306 in FIG. 9, a music CD may be used for such music. In this case, anindication is shown on the display 18A to notify the user of a need fora music CD. When the user loads a music CD into the disk loading unit 22instead of a CD-ROM in which programs are stored, pieces of musicrecorded on the music CD are shuffled to randomly select a piece ofmusic for allowing the user to enjoy the vibribbon game endlessly. Thevibribbon game will be described later in detail.

Obviously, if a music CD is loaded in the music player 298 in advance,the vibribbon game can be executed when the “endless mode” is selectedwithout removing the optical disk 20 in which the program and data ofthe vibribbon game are recorded from the disk loading unit 22. In thiscase, the real-time characteristics of the game is further improved.Specifically, when a music CD is loaded in the music player 298 inadvance, pieces of music recorded on the music CD are shuffled at apoint (step) instructed by the program to randomly select a piece ofmusic and the selected piece of music is read and stored into theentertainment apparatus 12 through the music player 298 substantially inreal time.

Each time either the up button 110 a or down button 110 d is pressedwhen the game selection screen 306 is displayed, the decagonal object308 and names of selectable game modes or the like are rotated as shownon a game selection screen 310 in FIG. 10 to allow selection of otherdesired games. In FIG. 9, a “Speed” mode in which music is played at afast tempo is highlighted. In this state, the user can select the“Speed” mode by pressing the decision button 112 b.

When an “Exit” mode is selected on the game selection screen 306 in FIG.8 or 9, the process returns to the vibribbon game start screen 300 shownin FIG. 6.

The present embodiment is on an assumption that the ◯ button 112 b as adecision button is pressed at step S9 in the state of the game selectionscreen 306 (see FIG. 9). When the decision is made, the vibribbon gamein the “easy” mode is started, and a game process at step S10 isperformed.

A detailed flow of the game process at step S10 is shown in FIG. 11.

First, it is checked at step S21 whether an initial process of a gameprocess as described at the next step S22 has been carried out or not.

In the initial process of the game process at step S22, three types ofcharacter objects 401, 402 and 403 shown in FIG. 12, four types ofobstacle objects 411, 412, 413 and 414 shown in FIG. 13 and a movementpath (also referred to as “virtual road”) object 420 shown in FIG. 14stored in the optical disk 20 are read and stored in the main memory 253using a world coordinate system.

The character objects 401, 402 and 403 are modified representations ofanimals such as a rabbit, a frog and a snake. The obstacle objects 411,412, 413 and 414 are modifications of a quadrangle (a square or boxyshape), a circle, a V-shape (an inverted triangle) and a zigzag (asymbol for a resistor), respectively. Further, the virtual road object420 is a virtual road (a three-dimensional line drawing image) on whichthe character objects 401, 402 and 403 move. The obstacle objects 411,412, 413 and 414 generated in accordance with results of sound analysis(audio analysis) as described later are inserted in the virtual roadobject 420.

In this case, each of the character objects 401, 402 and 403, theobstacle objects 411, 412, 413 and 414, and the virtual road object 420is basically constituted by basic objects 415 as convex shape models(convex polyhedral models) in the form of an elongate rectangularparallelepiped as shown in FIG. 15. FIG. 15 schematically shows aconfiguration of the obstacle object 411 as an example. Magnification,reduction, coordinate transform (including movement) and the like on thebasic object 415 can be performed by the GTE 261.

As shown in FIG. 15, the polygons that constitute the character objects401, 402 and 403, the obstacle objects 411, 412, 413 and 414 and thevirtual road object 420 are separated into polygonal components in theform of, for example, a quadrangle (or a triangle) that constitute thebasic objects 415. Those polygons are defined by the three-dimensionalcoordinates of the vertices thereof and colors of those vertices and arestored in a predetermined area of the main memory 253 (an area forstoring the character objects 401, 402 and 403, the obstacle objects411, 412, 413 and 414 and the virtual road object 420).

In the present embodiment, the color is stored as white (for example,the tone values of R (red), G (green) and B (blue) are stored as R(red)=G (green)=B (blue)=255 when the brightness levels are representedby eight bits). Obviously, a different color may be used.

Further, in the initial process at step S22, a table 416 ofcorrespondence between control buttons PB and the obstacle objects 411,412, 413 and 414 (a control buttons/obstacle objects correspondencetable) for executing the vibribbon game schematically shown in FIG. 16is read from the optical disk 20 and stored in a predetermined area ofthe main memory 253 (a control buttons/obstacle objects correspondencetable storing area).

As shown in FIG. 16, on the correspondence table 416, the L1 button 114a, R1 button 116 a, up button 110 a and Δ button 112 a are assigned tothe obstacle objects 411, 412, 413 and 414, respectively.

Furthermore, in the initial process at step S22, flags as describedlater (an NG flag F1 and etc.), a register (character object statusregister) 456 and the like are set in an initial state (which will bealso described later).

After the above-described initial process at step S22 is completed, itis checked whether there is any further music data in the buffer 283 ornot in a process at step S23. If there is no further music data in thebuffer 283, it is checked whether a game for one piece of music data hasbeen completed or not at step S24. If the game for one piece of musicdata has not been completed, for example, music data for one piece ofmusic in the “easy mode” is read from the optical disk 20 and the readmusic data is stored in the main memory 253 at step S25. Alternatively,the data may be stored in the buffer 283.

In the above-described endless mode, for example, in the process at stepS25, music data for one piece of music is read from a CD or the likeloaded in the music player 298 and the read music data is stored in themain memory 253.

Next, an audio signal analyzing process at step S26 and a line drawingdisplay updating process at step S27 are performed in parallel todisplay a game image on the screen of the display 18A.

FIG. 17 shows a flow chart of the audio signal analyzing process (audiosignal analyzing means) at step S26.

In a process at step S51, it is determined whether the music data for apredetermined time of reproduction have been read or not by determiningwhether the music data is stored in the buffer 283.

If the music data is not stored in the buffer 283, at step S52, themusic data for one piece of music is read for the predetermined timefrom the beginning thereof and is written in the buffer 283. In thepresent embodiment, the predetermined time is eight seconds (exactly,eight seconds plus marginal time) that is time required for a relativemovement of the virtual road object 420 for a distance of one screenfrom the upper right side to the lower left side of the screen.

A description will now be made on a process of analyzing an audio signalto determine the occurrence of an obstacle object. The music data foreight seconds stored in the buffer 283 (an area for storing music datafor the predetermined time) is divided into a predetermined number ofparts each of which lasts for a very short period of time (16 parts eachof which lasts 0.5 sec. in the present embodiment).

In this case, in order to divide an audio signal (also referred to as“music data”) into very small periods of time each of which is 0.5 sec.in the present embodiment, music data for 0.5 sec. is read from thebuffer 283 at step S53.

At step S54, a register i is incremented by one (i←i+1) as a countingparameter for the reading operation.

The music data for the very short period is sampled at a certainsampling frequency at step S55, and a frequency spectrum is extracted atstep S56. That is, a fast Fourier transform process is performed. Thesampling may be followed by a band-pass filtering process in an audiofrequency band to eliminate noises.

In a process at step S57, three (this number of peak values may beappropriately changed) peak values (peak values representing theloudness of sounds) are detected in each of a frequency range equal toor higher than a predetermined frequency fc (this frequency may bevaried at random) and a frequency range lower than the same in theextracted frequency spectrum. At step S58, the detected peak values inthe frequency spectrum are arranged in the order of magnitude in each ofthe frequency range lower than the predetermined frequency fc and thefrequency range equal to or higher than the predetermined frequency fcto determine respective orders of arrangement of the three peak valuesup to the third peak.

For example, assuming that f11, f12 and f13 represent the three peakfrequencies lower than the predetermined frequency fc in an ascendingorder and that f4, f5 and f6 represent the three peak frequencies equalto or higher than the predetermined frequency fc in an ascending order.Then, since there are six combinations of peak frequencies in each ofthe frequency region equal to or higher than fc and the frequency regionlower than fc, there are 36 possible orders P of arrangement of peakfrequencies in total.

At step S59, reference is made to a table 428 of correspondence betweenthe peak frequency arranging orders P and the obstacle objects 411, 412,413 and 414 (a table of correspondence between results of audio signalanalysis and obstacle objects to be generated). At step S60, it isdecided which of the obstacle objects 411, 412, 413 and 414 is to begenerated based on the present frequency analysis.

As shown in FIG. 18, for example, it is decided to generate the obstacleobject 411 when the peak frequency arranging order P=[f11, f12, f13,fh1, fh2, fh3], and it is decided to generate the obstacle object 414when P=[f13, f12, f11, fh3, fh2, fh1].

The order of generation of a plurality of obstacle objects may bedecided based on a result of one frequency analysis.

The audio signal analyzing process at steps S53 through S60 is merely anexample of an audio signal analyzing process performed using thefrequency axis. Alternatively, the audio signal analyzing process can beperformed by using the time axis. Specifically, music data may bedivided into parts each having a predetermined period of time, e.g., 0.5sec. Then, peak values of amplitudes of sounds in a divided period oftime on the time axis may be extracted in a descending order. Then,gradients Q between adjoining peaks of amplitudes on the time axis maybe calculated, and a correspondence table 429 may be provided aspermutational combinations of the gradients Q, as shown in FIG. 19. Forexample, it is decided to select the obstacle object 411 when acombination of gradients Q between peaks consists of four consecutivepositive gradients.

In the audio signal analyzing process, the order of appearance of theobstacle objects 411, 412, 413 and 414 may be determined in advancebased on data in a table of contents of a CD (the number of pieces ofmusic, playing times thereof, logical addresses of the pieces of music,etc.) instead of the audio signal itself, for example, in the endlessmode.

A line drawing display updating process at step S27 (see FIG. 11) isthen performed, and processes at steps S61 and S62 are performed inparallel with the line drawing display updating process. The process atstep S61 repeats the processes from steps S53 to S60 until the value inthe register i associated with the counter parameter set at step S54becomes an i-value=16 (a value corresponding to eight sec. perioddescribed above). When i=16, the value in the register i associated withthe counter parameter is set at an i-value=0 at step S62. At this time,all of the music data for eight sec. in the buffer 283 (the area forstoring music data for a predetermined time) is read, and the processproceeds to step S27 (see FIG. 11).

FIG. 20 is a flow chart of the process of updating line drawing display(including the initial display) at step S27.

At step S71, a single line drawing image in the form of a substantiallystraight line (which is split straight lines actually) extending fromthe lower left end to the upper right end of the screen of the display18A of the display monitor 18 is generated by the GTE 261 from thevirtual road object 420 (see FIG. 14). The GPU 262 draws the image ineither of drawing regions 265 (i.e., two drawing regions 265A and 265B),e.g., the drawing region 265A in the schematic diagram of the framebuffer 263 shown in FIG. 21. The frame buffer 263 has a size of 1024pixels and 512 pixels, for example, in x- and y-directions respectivelyand functions as a two-buffer having drawing regions 265A and 265B eachof which is formed by 256 pixels×240 pixels, for example.

At step S72, as will be described later with reference to a drawing,line drawing images of the obstacle objects 411, 412, 413 and 414 whichare non-linear line drawing images determined based on a result ofanalysis of an audio signal at step S58 are similarly drawn in thedrawing region 265A of the frame buffer 263 such that they are insertedin the single substantially linear line drawing image in locations deepin the screen on right side thereof in the order in which they areanalyzed. Thus, a linear line drawing image and non-linear line drawingimages are synthesized.

Further, at step S73, a line drawing image of the predeterminedcharacter object 401 (see FIG. 12) is similarly drawn in the drawingregion 265A of the frame buffer 263 such that it is drawn on the singlesubstantially linear line drawing image having the non-linear linedrawing images in the vicinity of the left end of the screen to besynthesized with the same. The selection of any of the character objects401, 402, 403, etc. is carried out in accordance with the contents of acharacter object status register 456 which will be described later withreference to FIG. 32. When the game is started, in the above-describedinitializing process at step S22, data associated with the characterobject 401 is set as the contents (data) of the character object statusregister 456.

In the processes at steps S71, S72 and S73, drawing is performed byrendering processes on basic objects 415 in the form of an elongaterectangular parallelepiped (see FIG. 15) that respectively constitutethe virtual road object 420 comprising the substantially linear linedrawing image, the obstacle objects 411, 412, 413 and 414 comprisingnon-linear line drawing images and the character object 401 comprising anon-linear line drawing image. The rendering processes include acoordinate transform from the world coordinate system to a cameracoordinate system, a perspective transform to transform the coordinatesystem further into a screen coordinate system, processes on hiddensurfaces and coloring processes on the polygons (a scaling process isalso performed appropriately).

FIG. 22 schematically shows a process performed on the virtual roadobject 420 constituted by a basic object 415 before it is disposed in acamera coordinate system xyz and in a screen coordinate system xy (anx-y plane). Thus, in the example shown in FIG. 22, a singlethree-dimensional line drawing image in the form of a straight line isdisplayed such that it extends from the lower left end on the front sideof the screen (x-y plane) of the display 18A to the upper right end onthe further side of the screen.

For easier understanding, a description will be made on athree-dimensional image which is displayed on the display 18A based ondrawing data read from the drawing region 265B in which drawing has beenperformed in advance, of the drawing regions 265A and 265B.

FIG. 23 shows a three-dimensional line drawing image 430 which is readfrom the drawing region 265B and displayed on the screen of the display18A with a coloring process and the like performed thereon by the GPU262.

The three-dimensional line drawing image 430 is an image in which acharacter object line drawing image 401Ia is placed on the left end of avirtual road object line drawing image 420Ia formed by pieces of linedrawing images i.e., vibrating basic objects 415. The vibrating objects415 are separate from each other.

At step S74, the virtual road object line drawing image 420Ia (a virtualline drawing image having obstacle object line drawing images insertedtherein in a case wherein obstacle object line drawing images arepresent) is drawn such that it moves a predetermined distance in thedirection of the arrow E at a predetermined time interval, e.g., eachtime the screen is updated (every {fraction (1/30)} sec. in the case ofan NTSC system). At the same time, components that form the characterobject line drawing image 401Ia such as the arms, legs, etc. of amodified rabbit in this case are drawn such that they are alternatelymoved back and forth to provide an image in which the character objectline drawing image 401Ia seems as if it is in a relative movement(running) in the direction of the arrow F on the screen.

The three-dimensional line drawing image 430 shown in FIG. 23 is animage in which only the character object line drawing image 401Ia andthe virtual road object line drawing image 420Ia are displayed. Linedrawing images associated with obstacle objects that are in accordancewith results of frequency analysis are displayed based on results offrequency analysis after the three-dimensional line drawing image 430 isdisplayed.

In FIG. 23, a reference numeral 415I represents a line drawing image ofa basic object 415 (a basic object line drawing image). In practice,since a basic object 415 is quite thin, only edge lines of the polygonsthat constitute the object are drawn in white.

Therefore, the three-dimensional line drawing image 430 in the examplein FIG. 23 is a quite simple monochromatic image (a monochromaticpicture, in practice) in which the background is in black and linedrawing portions formed by edge lines of polygons are in white.

In the present embodiment, the time required for the right end of thevirtual road object line drawing image 420I to move to the left end ofthe three-dimensional line drawing image 430 is set at eight sec. asdescribed above.

In practice, when an obstacle object line drawing image as describedlater appears in the virtual road object line drawing image 420Ia on thescreen of the display 18A of the monitor 18, the user (game player) canperform operations on the control buttons PB as prescribed in thecontrol buttons/obstacle objects correspondence table 416 in FIG. 16 atpredetermined timing according to various elements of music outputtedfrom the speaker 18B of the monitor 18 or headphones to clear theobstacle object line drawing image. The terms “clear” indicates a statein which the character object line drawing image gets over an obstacleobject line drawing image or rolls over the same to move relative to thesame at proper timing according to music. When the user fails to performa prescribed operation on the control buttons PB at predetermined timingto enter a non-clear state, a particular image is generated as describedlater. The vibribbon game proceeds in such a manner.

In the present embodiment, a clear state is determined at step S74 basedon a state of an NG flag F1 as described later. When the NG flag F1 is 0(the clear state), a small vibration imparting process is performed toimpart small vibrations (relatively small vibrations) to the nextthree-dimensional line drawing image to be drawn. When the NG flag F1 is1 (the non-clear state or NG state), a big vibration imparting processis performed at step S77 to impart big vibrations (relatively bigvibrations) to the next three-dimensional line drawing image to bedrawn.

In general, small vibrations give the user (operator) a pleasant feeland a sense of rhythm, and big vibrations give the user (operator) asurprise and the like. The speaker 18B generates pleasant music with asense of rhythm synchronously with small vibrations and generates soundssuch as loud blasts synchronously with big vibrations. The music may bemuted.

In this case, the terms “small vibrations” and “big vibrations”represent a difference in the degree of vibrations. In the presentembodiment, the term “small vibrations” indicates a level of vibrations(small vibrations) which does not make it difficult for the user torecognize the original shape of an object. The term “big vibrations”indicates a level of vibrations (big vibrations) which makes itdifficult for the user to recognize the original shape of an object.

When a big vibration imparting process is performed at step S77, the NGflag F1 is reset to F1←0 (F=0) (flag is taken down) at step S78.

At step S79, a new three-dimensional line drawing image which has beensubjected to a vibration imparting process (a process of imparting smallor big vibrations) is drawn in the drawing region 265A in which drawingis presently performed instead of the drawing region 265B which ispresently being read for display by a process at step S78.

The vibration process at steps S76 and S77 will now be described.

The vibration process is a process in which after a random number isadded to each of the vertices of the polygons that form each of basicobjects 415 which are pieces of line drawing images forming all objectsprovided in a three-dimensional space, images constituted by only edgelines of the polygons are drawn again.

In a mathematical description, relatively small random numbers RDS aregenerated for the small vibration process, and relatively big randomnumbers RDB are generated for the big vibration process. When the NGflag F1 is 0, relatively small random numbers RDS (Δxs, Δys, Δzs) areadded to the coordinates (x, y, z) of the respective vertices of a basicobject 415 to transform the vertex coordinates into vertex coordinates(x+Δxs, y+Δys, z+Δzs), and straight lines are drawn between thetransformed vertex coordinates to define the edge lines of a newpolygon.

When the NG flag F1 is 1, relatively big random numbers RDB (Δxb, Δyb,Δzb) are added to the coordinates (x, y, z) of the respective verticesto transform the vertex coordinates into vertex coordinates (x+Δxb,y+Δyb, z+Δzb), and straight lines are drawn between the transformedvertex coordinates to define the edge lines of a new polygon.

Referring now to FIG. 24 for a graphical description, the smallvibration process creates a basic object 415 a by slightly moving (i.e.,rotating, enlarging or displacing) a basic object 415 which is initiallydrawn in a quantity represented by the arrows SV and SV′ in thethree-dimensional space, and the big vibration process creates a basicobject 415 b by moving the basic object 415 in a larger quantityrepresented by the arrows LV and LV′ in the three-dimensional space.

At step S77, the three-dimensional object to which vibrations have beenimparted is drawn in the drawing region 265 (265A or 265B) in which nodrawing is presently performed. When it is drawn in the drawing region265, since no texture is applied to the surfaces of the polygon thatconstitutes the basic object 415, no change occurs in the quality andthe feel of the material of the basic object 415 even if it is enlargedor reduced. In other words, an advantage is achieved in that thesimplicity of the image is not deteriorated even if it is enlarged orreduced.

The three-dimensional line drawing image 430 shown in FIG. 23 is animage in which small vibrations are imparted to each of the basic objectline drawing images 415Ia.

At step S79, the three-dimensional line drawing image to whichvibrations have been imparted is drawn in the drawing region 265B whichis not presently being displayed. At step S80, display is presented fromthe drawing region 265A in which drawing has already been completed. Asdescribed above, the display process at step S80 and other processes areperformed in parallel. The other processes indicate processes at stepS26 (steps S51 through S60) and at steps S71 through S79 and processesfrom step S28 through step S23 up to step S26.

For convenience in understanding, a description will now be made withreference to FIGS. 25 through 32 on a three-dimensional line drawingimage displayed on the screen of the display 18A and the progress of agame.

FIG. 25 shows a three-dimensional line drawing image 432 having smallvibrations imparted thereto which is obtained after the game processflow at step S10 shown in FIG. 11 is repeated for several seconds.

While separate pieces of line drawing images that form a game characterrepresent the character as if it is running in the three-dimensionalline drawing image 432, the entire image is presented as an image inwhich an obstacle object line drawing image 411Ib, an obstacle objectline drawing image 414Ib and an obstacle object line drawing image 412Ibinserted in a virtual road object line drawing image 420Ib aresequentially moved from the further right side of the screen to thefront left side of the screen (in the direction of the arrow E) relativeto a character object line drawing image 401Ib which is relativelystationary in the vicinity of the left end of the screen.

Specifically, in the three-dimensional line drawing image 432,figuratively speaking, a rabbit (the character object line drawing image401Ib) seems as if it is running while moving up and down at thepositions of a quadrangular obstacle object (the obstacle object linedrawing image 411Ib), a zigzag obstacle object (the obstacle object linedrawing image 414Ib), a V-shaped obstacle object (the obstacle objectline drawing image 414Ib) and a circular obstacle object (the obstacleobject line drawing image 412Ib) which are moving toward the rabbit.

In a three-dimensional line drawing image 434 shown in FIG. 26, when theL1 button 114 a is pressed at predetermined timing (in a predeterminedrange) in response to a movement of an obstacle object line drawingimage 411Ic toward the front left end of the screen and a resultantincrease in the size of its quadrangular configuration, a characterobject line drawing image 401Ic gets over the quadrangular obstacleobject line drawing image 411Ic in a manner like leapfrog. Thus, thequadrangular obstacle object line drawing image 411Ic can be cleared.

At this time, a virtual road object line drawing image 420Ic, anobstacle object line drawing image 414Ic, an obstacle object linedrawing image 413Ic and an obstacle object line drawing image 412Ic alsomove in the direction of the arrow E while gradually increasing in size.

In a three-dimensional line drawing image 436 shown in FIG. 27, when theΔ button 112 a is pressed at predetermined timing (in a predeterminedrange) in response to a movement of an obstacle object line drawingimage 414Id toward the front left end of the screen and a resultantincrease in the size of its zigzag configuration, a character objectline drawing image 401Id moves over the zigzag obstacle object linedrawing image 414Id by making a so-called forward roll on the same.Thus, the zigzag obstacle object line drawing image 414Id can becleared.

At this time, a virtual road object line drawing image 420Id, anobstacle object line drawing image 413Id and an obstacle object linedrawing image 412Id also move in the direction of the arrow E whilegradually increasing in size.

In a three-dimensional line drawing image 438 shown in FIG. 28, when theup button 110 a is pressed at predetermined timing (in a predeterminedrange) in response to a movement of an obstacle object line drawingimage 413Ie toward the front left end of the screen and a resultantincrease in the size of its V-shaped configuration, a character objectline drawing image 401Ie moves over the V-shaped obstacle object linedrawing image 413Ie in such a matter that it strides over the same.Thus, the V-shaped obstacle object line drawing image 413Ie can becleared.

At this time, a virtual road object line drawing image 420Ie and anobstacle object line drawing image 412Ie also move in the direction ofthe arrow E while gradually increasing in size.

In the three-dimensional line drawing image 438, a new obstacle objectline drawing image 414Ie which is generated as a result of an audiosignal analyzing process performed concurrently with the display processis drawn on the right end of the virtual road object line drawing image420Ie.

In a three-dimensional line drawing image 440 shown in FIG. 29, when theR1 button 116 a is pressed at predetermined timing (in a predeterminedrange) in response to a movement of an obstacle object line drawingimage 412If toward the front left end of the screen and a resultantincrease in the size of its circular configuration, a character objectline drawing image 401If moves in the circular obstacle object linedrawing image 412If in such a manner that it seems like walking. Thus,the circular obstacle object line drawing image 412If can be cleared.

At this time, a virtual road object line drawing image 420If, anobstacle object line drawing image 414If and a newly generated obstacleobject line drawing image 413If also move in the direction of the arrowE while gradually increasing in size.

FIGS. 26 through 29 show line drawing images in which the characterobject 401 clears the obstacle objects 411, 412, 413 and 414,respectively.

FIG. 30 shows a three-dimensional line drawing image 450 that appearsimmediately after a so-called non-clear state which occurs when the L1button 114 a is not pressed at the predetermined timing (in thepredetermined range) relative to the obstacle object line drawing image411 b in the display of the three-dimensional line drawing image 432shown in FIG. 25 or when a control button PB other than the L1 button114 a is pressed even though at the predetermined timing (in thepredetermined range).

As shown in FIG. 30, big vibrations (explosive vibrations) described inthe process at step S77 are imparted to each of basic object linedrawing images 415Ig that form an obstacle object line drawing image411Ig to display it as an image of broken pieces. Such big vibrationsalso affect a character object line drawing image 401Ig and a virtualroad object line drawing image 420Ig in the vicinity of the same. Asshown in FIG. 30, this results in an image in which relatively bigvibrations are imparted also to basic object line drawing images 401Igthat form the character object line drawing image 401Ig and virtual roadobject line drawing image 420Ig, although the vibrations are stillcategorized as small vibrations according to the process at step S76.

At this time, vibrations may be imparted to the joysticks 44 and 46through the motor driver 150 and motor 130.

The three-dimensional line drawing image 450 including the broken objectshown in FIG. 30 clearly indicates that the user could not clear theobstacle object line drawing image 411Ig (the non-clear or NG state).

FIG. 31 shows a three-dimensional line drawing image 452 that appearswithin a predetermined time (e.g., within one second) after a failure inclearing the obstacle object line drawing image 411Ig.

As shown in FIG. 31, when the obstacle object line drawing image 411Ib(or any one of the other obstacle object line drawing images 414Ib,413Ib and 412Ib) shown in FIG. 25 was not cleared, an image appears inwhich vibrations have been imparted to enhance small vibrationsslightly. Further, in such a non-clear state, the moving speed of thevirtual road object line drawing image 420Ig in the direction of thearrow E may be increased to reduce predetermined timing (a predeterminedrange) that allow a character object 401Ih to clear an obstacle object414Ih, thereby increasing the difficulty of the game.

FIG. 32 is a character status table 454 showing changes in the statuses(metaphorically speaking, degeneration and evolution) of the characterobjects 401, 402 and 403 shown in FIG. 12 in the “easy” mode.

As shown in the character status table 454, the character object thatappears first (at the time when the game is started) in the “easy mode”of the vibribbon game is the character object 401 which is amodification of a rabbit and to which very slight vibrations (smallvibrations at step S76) are imparted. When the character object 401fails to clear any one of the obstacle objects 411, 412, 413 and 414,the above-described big vibrations are imparted to the character objectto break up the same, and a character object 401′ having slightly biggervibrations appears thereafter.

When the character object 401′ having bigger vibrations (vibrations thatstill leave the original shape as described at step S76) fails to clearan obstacle object again, the above-described big vibrations areimparted to break up the same and to cause it to change (transformitself) into a character object 402 which is a modification of a frogand to which very small vibrations are imparted.

When failures in clearing are similarly repeated, the change of thecharacter object is repeated. Specifically, the above-described bigvibrations are imparted to break up the character object 402 to changeit into a character object 402′ to which slightly bigger vibrations areimparted. Then, the character object 402′ is caused to transform itselfinto a character object 403 which is a modification of a snake and towhich still smaller vibrations are imparted. Thereafter, the characterobject 403 is changed to a character object 403′ to which slightlybigger vibrations are imparted. In this manner, each time the characterobject fails in clearing an obstacle object, the appearance of thecharacter object gets miserable. In the end, when the obstacle objects411, 412, 413 and 414 can not be cleared over a predetermined number oftrials, that is, when the character object fails in clearing an obstacleobject after the character object is changed to the character object403′, the game is terminated, i.e., the game is over.

Even when the character object 401 once changes (degenerates) in thedirection of the arrow B, i.e., when the character object 401sequentially changes to the character objects 401′, 402, 402′, 403 and403′, changes in the direction of the arrow F that is opposite to thedirection of the arrow B (evolution) occurs if the clear stateconsecutively occurs or the probability of clearance increasesthereafter. For example, re-transformation from the character object 403into the character object 402′ and the like can occur.

Algorithm for defining what state of clearance triggers a transformationand so on is determined in advance for each of the game modes, and thenumber and pattern of such clear states are prescribed in the relevantprogram.

When a piece of music is terminated while the character object is in anyof the states represented by 401, 401′, 402, 402′, 403 and 403′, thegame mode is terminated in a clear state, and a point is displayed inaccordance with the states of clearance of the obstacle objects 411,412, 413 and 414 at that time.

The character object statuses 401, 401′, 402, 402′, 403 and 403′ arestored in a register in the CPU 251 (a character object status register(character object status storing region) 456 schematically shown in FIG.32) as character object statuses. When the game is started in the “easy”mode, the contents of the character object status register 456 are datarepresenting the character object 401.

A description has been made above on the three-dimensional line drawingimage displayed on the screen of the display 18A and the progress of thegame in accordance with the manual controller 16.

A description will now be made on the progress of the game in relationto the flow chart shown in FIG. 11.

When the three-dimensional line drawing image 432 or the like shown inFIGS. 25 through 31 is shown, e.g., when the three-dimensional linedrawing image 432 shown in FIG. 25 is displayed, it is checked at stepS28 whether any control button PB has been manipulated.

If no manipulation is determined, it is checked at step S29 whether thecharacter object line drawing image 401Ib has reached a predeterminedposition of the obstacle object line drawing image 411Ib, e.g., theleading position of the obstacle object line drawing image 411Ib. If thecharacter object line drawing image 401Ib has not reached thepredetermined position of the obstacle object line drawing image 411Ib,NG flag F1 is set at 0 at step S30 because it is not an NG state, andprocesses at step S23 and the subsequent steps are performed, i.e., theaudio signal analyzing process at step S26 and the line drawing displayupdating process at step S27 are performed if there is any further musicdata.

During the line drawing display updating process at step S27, displaywith small vibrations is maintained because F1=0 at the determination ofthe NG flag F1 at step S75 (see FIG. 20).

When it is determined at step S28 that a control button PB has beenmanipulated, it is determined at step S31 whether the obstacle objecthas been cleared. Specifically, it is determined with reference to apredetermined pixel-number table (not shown) and the controlbuttons/obstacle objects correspondence table 416 shown in FIG. 16whether a predetermined part of the character object line drawing image401Ib, e.g., the part of a front leg is located within a predeterminedrange from a predetermined position of the obstacle object line drawingimage 411Ib (e.g., the leading position of the obstacle object linedrawing image 411Ib) at the time of manipulation (the determination isactually made based on a certain number of pixels) and whether theappropriate control button PB, i.e., the L1 button 114 a to get over theobstacle object line drawing image 411Ib has been manipulated or not.

When both of these conditions are satisfied, at step S31, it isdetermined that the obstacle object is cleared. At step S32, the NG flagF1 is set in a state representing successful clearance, i.e., F1←0. Whenthe obstacle objects 411, 412, 413 and 414 are cleared, points are addedto an unillustrated point register.

When either of those conditions is not satisfied, step S31 determinesthat the obstacle object is not cleared. At step S33, the NG flag F1 isset in a state representing unsuccessful clearance, i.e., F1←1.

When it is determined at step S29 that no control button PB has beenmanipulated, the NG flag F1 is set in the F1←1 (NG) state based on ajudgement that the manipulation of the control buttons PB has beendelayed even if the character object line drawing image 401Ib or thelike has reached a predetermined position of the obstacle object linedrawing image 411Ib or the like.

After the process (F1←0) at step S33, it is determined at step S34whether the normalization of the character objects 401, 402 and 403(i.e., a change in the direction of the arrow F in FIG. 32) is possiblein the present state of display. For example, the term “normalization”means a change of the character object 401′ (see FIG. 32) into thecharacter object 401 having smaller vibrations and a change of thecharacter object 402 into the character object 401′ in the direction ofthe arrow F.

When the determination at step S34 is YES, in other words, when it isdetermined with reference to the data in the character object statusregister 456 that the character object is in any of the statusesindicated by the 401′, 402, 402′, 403 and 403′ excluding 401, at stepS36, the data of the character object status register 456 is rewrittenwith data representing a character object in the direction of the arrowF.

Obviously, the determination at step S34 is NO when the data of thecharacter object status register 456 is data representing the characterobject 401.

As assumed from the processes at steps S32, S34 and S36, after theprocess (F1←1) at step S32, it is determined at step S35 whether thedeterioration of the character objects 401, 402 and 403 (i.e., a changein the direction of the arrow B in FIG. 32) is possible in the presentstate of display. For example, the term “deterioration” means atransformation of the character object 401′ (see FIG. 32) into thecharacter object 402 and a transformation of the character object 402into the character object 402′ having bigger vibrations in the directionof the arrow B.

When the determination at step S35 is YES, in other words, when it isdetermined with reference to the data in the character object statusregister 456 that the character object is in any of the statusesindicated by the 401, 401′, 402, 402′ and 403, at step S36, the data ofthe character object status register 456 is rewritten with datarepresenting a character object in the direction of the arrow B.

When the determination at step S35 is NO, the data of the characterobject status register 456 is data representing the character object403′. Then, the process proceeds to step S11 (see FIG. 5).

The process proceeds to step S11 as well when it is determined at stepS24 that the game has been finished for one piece of music.

FIGS. 33, 34 and 35 respectively show ending screens 457, 458 and 460used in the process at step S11 of the termination process at step S12.

Specifically, when the determination at step S35 is negative, the endingscreen 457 shown in FIG. 33 is displayed.

On the ending screen 457, characters that read “game over! (meaning theend of the game)”, “once more? (asking whether the player wishes to playthe game once more)”, “Yes” and “No” are displayed with small vibrationsimparted thereto. When the ◯ button 112 b is pressed in this state, thegame can be played again. That is, step S12 results in a negativedetermination and the game process at step S10 is started. Then, thegame selection screen 306 shown in FIG. 9 is displayed.

When “Exit” is selected on the game selection screen 306, the startscreen 300 shown in FIG. 6 appears.

The ending screen 458 shown in FIG. 34 is a screen that appears when“No” is selected on the ending screen 457 shown in FIG. 33 using thedown button 110 c. When the ◯ button 112 b is pressed in this state,step S12 results in a positive determination. Then, the start screen 300shown in FIG. 6 is displayed.

When step S24 results in a positive determination, the ending screen(game clear screen) 460 shown in FIG. 35 is displayed.

On the ending screen 460, characters that read “clear!” and “your scoreis 1570” are displayed with small vibrations imparted thereto. When the◯ button 112 b is pressed in this state, the game can be played again.That is, step S12 results in a negative determination and the gameprocess at step S10 is started. Then, the game selection screen 306shown in FIG. 9 is displayed.

FIG. 36 shows a functional block diagram for image processing and audioprocessing according to the above-described embodiment.

Referring to FIG. 36, audio signal analyzing means 502 has audio signaldividing means 504 for dividing an audio signal read from the opticaldisk 20 or a music CD or the like at predetermined time intervals,sampling means 506 for sampling the audio signal divided at thepredetermined time intervals, frequency spectrum detecting means 508 fordetecting frequency spectra from the result of the sampling, peak valuedetecting means 510 for detecting a peak value of each of the detectedfrequency spectra or detecting a peak value of a signal directly fromthe result of the sampling, order determining means 512 for determininga certain order by processing the detected peak values and non-linearobject determining means (obstacle object determining means) 514 fordetermining the obstacle object 411 and the like based on the determinedorder.

The process of determining an order performed by the order determiningmeans 512 will be described below. In a process on the frequency axis(frequency analysis process) which uses the frequency spectrum detectingmeans 508, the detected peak values are categorized into peak values infrequency bands lower and higher than, for example, 500 Hz. Then, thedetected frequencies are arranged in the order of the magnitude of thepeak values in each of the high and low frequency bands. The arrangementof the detected frequency is used as the above order. In a process onthe time axis (amplitude analysis process) which does not use thefrequency spectrum detecting means 508, peak values adjacent to eachother on the time axis among the five greatest detected peak values areconnected. Then, the gradient (differential value) between a peak valueand the next peak value is defined as positive or negative. Thearrangement of the positive and negative gradients is used as the aboveorder.

The non-linear object determining means 514 refers to the table 428 or429 showing correspondence between results of audio signal analysis andobstacle objects to be generated, determines a predetermined non-linearobject (obstacle object) which is determined in advance in accordancewith an order decided as described above and transmits the same tonon-linear line drawing image generating means 516.

In the functional block diagram for image processing and audioprocessing in FIG. 36, linear line drawing image generating means 518and character object line drawing image generating means 520 areprovided as line drawing image generating means in addition to thenon-linear line drawing image generating means 516. In this case, thecharacter object line image drawing generating means 520 generates apredetermined character object line drawing image based on adetermination made by character object line drawing image changedetermining means 524 which determines a change to be made on acharacter object line drawing image from a result of monitoring suppliedby manipulation monitoring means 522 which monitors the manipulationtiming of a predetermined control button PB on the manual controller 16.

Movement imparting means 526 imparts a quantity of movement to thelinear line drawing image, non-linear line drawing image and characterobject line drawing image.

Vibration quantity determining means 528 determines a quantity ofvibration based on a result of monitoring performed by the manipulationmonitoring means 522.

Vibration imparting means 530 imparts different vibrations to each ofthe linear line drawing image, non-linear line drawing image andcharacter object line drawing image to which a quantity of movement hasbeen imparted based on the quantity of vibration determined by thevibration quantity determining means 528.

The linear line drawing image, non-linear line drawing image andcharacter object line drawing image to which movements and vibrationshave been imparted are synthesized by synthesis means 532 and are drawnin the frame buffer 263 by drawing means 534.

The image drawn in the frame buffer 263 is displayed on the screen ofthe display 18A under control of display control means 536 (GPU 262).

As described above, the entertainment system 10 according to the presentembodiment has the entertainment apparatus 12 for executing variousprograms, the manual controller 16 for inputting a manual controlrequest of a user to the entertainment apparatus 12 and the display 18Afor displaying an image outputted from the entertainment apparatus 12.The entertainment apparatus 12 has the audio signal analyzing means 502for analyzing an audio signal and the line drawing image generatingmeans 516, 518 and 520 for generating a substantially linear linedrawing image having a non-linear line drawing image portion on thedisplay monitor 18 by generating a substantially linear line drawingimage (420Ib or the like) and by inserting a non-linear line drawingportion (411Ib or the like) based on a result of the analysis of theaudio signal in the substantially linear line drawing image 420Ib or thelike and for generating a line drawing image of a character object(401Ib or the like) on the substantially linear line drawing imagehaving the non-linear line drawing image portion.

Specifically, a line drawing image of a character object (401Ib or thelike) is generated on a substantially linear line drawing image having anon-linear line drawing image portion which has been inserted based on aresult of analysis of an audio signal (411Ib and 420Ib or the like).This makes it possible to display a novel line drawing image accordingto music on the display 18A.

In this case, the movement imparting means 526 may move the line drawingimage of the character object (401Ib or the like) such that it makes arelative movement on the substantially linear line drawing image 420Ibhaving the non-linear line drawing image portion 411Ib, which makes itpossible to provide a more entertaining line drawing image.

Further, the character object line drawing image change determiningmeans (character object line drawing image changing means) 524 maychange the character object line drawing image 401Ib or the like to aline drawing image of a different character object (402Ib or the like)depending on how the character object line drawing image moves on thesubstantially linear line drawing image 420Ib or the like having thenon-linear line drawing image portion 411Ib or the like, which makes itpossible to provide a more entertaining line drawing image.

Furthermore, the vibration imparting means 530 may impart vibrations tothe substantially linear line drawing image 420Ib or the like having thenon-linear line drawing image portion 411Ib or the like and thecharacter object line drawing image 401Ib or the like, which makes itpossible to provide a quite entertaining line drawing image.

In this case, each of the line drawing images may be drawn as athree-dimensional line drawing image to provide a highly entertainingimage which is less likely to become tiresome.

An audio signal may be used which is supplied to the entertainmentapparatus 12 from a recording medium (the optical disk 20 or a music CD)or which is downloaded thereto as a result of communication.

Each of the above-described audio signal analyzing means 502, the linedrawing image generating means 516, 518 and 520, the movement impartingmeans 526, the character object line drawing image change determiningmeans (character object line drawing image changing means) 524 and thevibration imparting means 530 may be stored in a recording medium suchas the optical disk 20 as a program.

For example, the operation of the game of the present embodiment may bedescribed with reference to FIG. 26. The L1 button 114 a (thepredetermined control button PB on the manual controller 16) is pressedat predetermined timing to cause the character object line drawing image401Ic to clear the virtual road object line drawing image 420Ic havingthe obstacle object line drawing images 411Ic, 414Ic, 413Ic and 412Icwhich move from the further right side of the screen toward the frontleft side of the screen (in the direction of the arrow E).

In this case, when the player misses the timing for pressing the controlbutton PB or presses a control button PB of a wrong type, as shown inFIG. 30, the obstacle object line drawing image 411 to be clearedbecomes the obstacle object line drawing image 411Ig which is broken insuch a manner that the original shape is indistinct, and the characterobject 401 changes to the character object line drawing image 401Ighaving considerably big vibrations.

The game operated in such a manner can be regarded quite entertaining.

The FIGS. 37 through 39 illustrate screens of match games (competitiongames) played by two game players according to other embodiments of thepresent invention.

In FIG. 37, a three-dimensional line drawing image 470 comprising athree-dimensional line drawing image 472 and a three-dimensional linedrawing image 474 is displayed on a screen 468. A character object linedrawing image 401I displayed at the lower left part of the screen 468moves relative to an lower virtual road object line drawing image 420Iand a character object line drawing image 403I displayed at the upperleft part of the screen 468 moves relative to an upper virtual roadobject line drawing image 420I, respectively. Specifically, obstacleobject line drawing images 414I, 411I, 412I, 413I inserted in therespective virtual road object line drawing images 420I movesequentially from the right side to the left side in the directionsindicated by arrows E.

In this state, one game player controls the character object linedrawing image 403I in the upper three-dimensional line drawing image 472by manipulating one of the manual controllers 16 and the other gameplayer controls the character object line drawing image 401I in thelower three-dimensional line drawing image 474 by manipulating the otherof the manual controllers 16 to play a match game (competition game).

In this game, the upper three-dimensional line drawing image 472 isdisplayed in reddish color and the lower three-dimensional line drawingimage 474 is displayed in bluish color so that the game players canrecognize their own game courses at a glance.

Since the same music data is used for generating each of the upper gamecourse and the lower game course, types and orders of the generatedobstacle object line drawing images in the game courses are the same asshown in the screen 468 of FIG. 37.

An arc-shaped line drawing image 476I displayed at the lower part of thescreen 468 indicates a length (duration) for one piece of music. Thearc-shaped line drawing image 476I comprises a thick solid line drawingimage and a thin solid line drawing image. The thick solid line drawingimage indicates a part of a music piece which has already beenreproduced and the remaining thin solid line drawing image indicates apart of the music piece which has not yet been reproduced. That is, thegame players can recognize the progress of the music piece from theborder between the thick solid line drawing image and the thin solidline drawing image. When the arc-shaped line drawing image 476I becomesthe thick solid line drawing image in its entirety, the gamecorresponding to the music piece is completed. In this game, the colorof the thick solid line drawing image the color of the thin solid linedrawing image are different.

In this embodiment, unlike conventional match games (competition games),the screen is not divided for displaying game images for two gameplayers. Rather, the three-dimensional drawing images 472 and 474 forgame courses of game players are displayed in a different color in thesame screen 468, respectively. Accordingly, a new fresh music game canbe played by two game players at the same time.

In FIG. 37, the three-dimensional line drawing images 472 and 474 aredisplayed in substantially parallel in a different color. In FIG. 38,the three-dimensional line drawing images 482 and 484 are displayeddiagonally to cross each other in a different color.

As described above, the game can be played by two game players at thesame time using the same music piece. However, the game is not limitedto a type in which the game players compete simply for obtaining a scoreor the like. According to another embodiment of the present invention,if one game player clears a predetermined number of obstacle object linedrawing images successively, a barrier is generated momentarily on thenext obstacle object line drawing image to be cleared (or a part of thevirtual road object 420 is removed momentarily on the next obstacleobject line drawing image to be cleared) in the other game player's gamecourse. In this case, it is no longer possible for the other game playerto clear the next obstacle object line drawing image even if the othergame player is succeeded in pressing a predetermined control button at aproper predetermined timing.

For example, as shown in FIG. 39, when one game player clears manyobstacle object line drawing images successively, line drawing imagesare displayed radially like a firework around the character object linedrawing image 401I of one of the three-dimensional line drawing image490 in the three-dimensional line drawing image 488. At the same time, abarrier is displayed (or a part of the virtual road object 420 isremoved) on the next obstacle object line drawing image 414I of theother of the three-dimensional line drawing image 492 in thethree-dimensional line drawing image 488. Line drawing images are alsodisplayed radially around the obstacle object line drawing image 414I.If this occurs, the other game player can not clear the obstacle objectline drawing image 414I any more.

In this manner, the manipulation of one game player may affect the gamecourse of the other game player. Accordingly, a further amusing aspectcan be added to the match game to excite the feelings of the gameplayers.

The present invention is not limited to the above-described embodiments,and various configurations may obviously be employed without departingfrom the principle of the invention.

(1) For example, as an alternative example of the controlbuttons/obstacle objects correspondence table 416, i.e., so-called keyassignment shown in FIG. 16, an control buttons/obstacle objectscorrespondence table 416A shown in FIG. 40 may be stored in addition. Onthe control buttons/obstacle objects correspondence table 416A, eitherthe L1 button 114 a or L2 button 114 b, either the R1 button 116 a or R2button 116 b, the down button 110 d and the X button 112 c are assignedto the obstacle objects 411, 412, 413 and 414, respectively. Such anarrangement makes it possible to satisfy preference of a user (gameplayer) and the like.

(2) For example, an obstacle object 602 obtained by synthesizing theobstacle objects 414 and 412 with the synthesis means as shown in FIG.41 may be generated in the tables 428 and 429 of correspondence betweenresults of audio signal analysis and obstacle objects to be generatedshown in FIGS. 18 and 19, as the obstacle object generated based on theaudio signal analyzing process at step S26. The user may need to pressthe R1 button 116 a and X button 112 c simultaneously at predeterminedtiming (in a predetermined range) to allow the character object 401 orthe like to clear the obstacle object 602.

Various synthesized obstacle objects 604, 606, 608, 610 and 612 as shownin FIG. 42 may be generated (created) as synthesized obstacle objects inaddition to the synthesized obstacle object 602.

(3) In order to simplify the operation of the game, the nameregistration process may be omitted by displaying the game selectionscreen 306 shown in FIG. 9 without performing the name registrationprocess (step S7) described with reference to FIGS. 7 and 8 when thestart button 40 is pressed with the start screen 300 being displayed.

(4) Furthermore, a special movement may be added to the obstacle objects411, 412, 413 and 414 and the virtual road object 420. First, forexample, the moving speed of the virtual road object line drawing image420Ib can be abruptly changed by setting the game program accordingly inrelation to the element of music (a piece of music) or regardless of theelement of music. Second, for example, the speed of the obstacle objectdrawing image 412Ib in FIG. 25 may increase such that the obstacleobject drawing image 412Ib passes the obstacle object line drawing image413Ib located in front of the same. Third, as seen on athree-dimensional line drawing image 622 displayed on a screen 620 ofthe display 18A in FIG. 43, obstacle object line drawing images 602Iiaand 602Iib may be displayed such that they rotate to the right and (or)left about a virtual road object line drawing image 420Ii while movingin the direction of the arrow E. It is to be understood that thesemodifications for displaying images can be applicable to the abovedescribed match game for two game players.

As described above, the present invention makes it possible to display anovel line drawing image on a display screen or the like.

Further, according to the invention, a line drawing image of a characterobject is generated on a substantially linear line drawing image havinga non-linear line drawing image portion based on a result of audiosignal analysis. This makes it possible to display a novel line drawingimage on a display screen or the like according to music.

The invention further makes it possible to display a line drawing imagehaving vibrations on a display screen.

Games in which line drawing images are displayed on a screen can bewidely accepted by people in different generations including childrenand old people because they give a heartwarming feeling.

Each of the line drawing images may be drawn as a three-dimensional linedrawing image to provide a highly entertaining image which is lesslikely to become tiresome associated with music.

Next, an audio signal analyzing process according to another embodimentof the present invention will be described in the following explanations(A. BRIEF EXPLANATION, B. DETAILED EXPLANATION).

A. Brief Explanation

The audio signal analyzing process comprises the following four steps(steps A1 through A4).

A1: Reading an audio signal in the optical disk (music CD) 20 andstoring the read audio signal in the buffer (long buffer) 283 or themain memory 253

A2: Emphasizing attacks in the music (audio sound) expressed by theaudio signal stored in the long buffer 283

A3: Selecting audio events

A4: Shadowing unnecessary audio events from the selected audio eventsand determining the resulting audio events as the final events (eventshadowing)

Firstly, the process in step A1 will be described. Specifically, anaudio signal is read from a music CD or the like via the optical diskdrive 281 and the decoder 282. The read audio signal is stored in thelong buffer 283. The audio signal in the long buffer 283 is delayed fora predetermined period of time.

The delay time allows audio events in the read audio signal to bedetected and displayed as road parts such as the obstacle object linedrawing image 411I on the display 18A in synchronism with the output ofthe corresponding audio sound from the speaker 18B via a D/A converter(not shown) in the SPU 271.

That is, the delay time is sufficient for the CPU 251 to detectdistinctive attacks (hereinafter also referred to as the distinctivepoints or the potential events) in the audio signal for determining roadparts corresponding to the detected attacks in the audio signal.

The audio signal comprises a sinusoidal wave signal having a variablychanging value (the audio signal has different values on the time axis).Each of positive values and negative values extracted as a samplingvalue constitutes an audio event. That is, positive audio events andnegative audio events are alternately repeated in the audio signal. Inparticular, distinctive events (attacks) in the audio events in theaudio signal are referred to as the distinctive points or the potentialevents.

Next, the process in step A2 will be described. The audio signal ispreprocessed to emphasize the attacks in the music (audio sound). Theemphasized audio signal can be expressed by the ratio (Ps/Pl) of a shortterm power Ps to a long term power Pl in the audio signal. The shortterm power Pl is calculated based on a short term Ns before an analysispoint and the long term power Pl is calculated based on a long term Nlbefore the analysis point.

More specifically, a certain point in the audio signal is determined asthe analysis point. Then, a short period of time, for example, about 23ms before the analysis point is determined as the short term Ns.Similarly, a long period of time, for example, about 186 ms before theanalysis point is determined as the long term Nl.

Generally, a plurality of audio events are included in each of the shortterm Ns and long term Ls. The short term Ns and long term Nl are alsoreferred to as the short term block and a long term block, respectively.

The short term power Ps and the long term power Pl can be calculated inthe following manner. The short term power Ps is taken to be the sum ofthe squares of the short term block's sampling values. The long termpower Pl is taken to be the sum of the squares of the long term block'ssampling values.

The squares are used for emphasizing sampling values. For example, asampling value greater than 1 is made much greater by multiplyingitself. A sampling value smaller than 1 is made much smaller bymultiplying itself. Further, the squares are used for convertingnegative sampling values into positive sampling values which aresuitable as power values.

That is, the emphasized signal is the ratio of the present (short termpower Ps) to the recent past (long term power Pl). The long term powerPl is smallest at the start of an audio event, and rises as the evententers the long term block. As a result, the start of an audio event isboosted by the long term power Pl in the denominator, and this boosttapers off as the event persists, Therefore, this algorithm tends toemphasize attacks in the music.

Next, the process in step A3 will be described. Event selection iscontrolled by a “select period”. At most one event will be generated perselect period. The length of the select period determines the maximumevent rate.

In order to be considered for event selection, the emphasized signalmust be greater than a threshold value. After thresholding, the peakemphasized signal (short term power Ps/long them power Pl) during eachselect period is chosen as a potential event. The ratio of the shortterm power Ps to the long term power Pl of the potential event is its“peak ratio”.

Next, the process in step A4 will be described. To prevent overlappingroad parts on the screen of the display 18A, the game's geometrydictates a minimum spacing between potential events. Event shadowingdrops potential events that would violate this constraint. The remainingevents are defined as final events.

An event's time extent is its “road part period”. The minimum timebetween two events is taken to be two times the first event's road partperiod—this is called the event's “shadow period”. No event may occur inanother event's shadow period.

When a potential event is selected, the potential event is temporarilystored in a memory. If its shadow period does not pass before anotherpotential event is selected, the peak ratios of two events are comparedand the event with the smaller ratio is dropped. Thus, the potentialevent with the larger ratio is determined to be the final event.

In this manner, a final event signal (final event array) having a seriesof final events is generated. When each of the final events isreproduced, one road part is displayed. The shape of the road partdisplayed in each of the final events is determined based apredetermined sequence distribution or weight random distribution.

In summary, according the audio analyzing process, the delay buffergives time for analysis and graphic display in step A1, the emphasisalgorithm highlights interesting events in the audio signal in step A2,event selection produces events with a desired maximum event rate instep A3, and, event shadowing drops events that violate spacingconstraints in step A4.

The audio analyzing process comprising the combination of these stepscan be effectively performed to generate interesting events from anaudio signal.

B. Detailed Explanation

Next, each process performed in steps A1 through A4 will be describedspecifically in the following sections (B1 Object, B2 Brief explanationof waveform processing, B3 Detailed explanation of waveform processing(B3a Emphasize process, B3b Event selection process, B3c Shadowingprocess)) with reference to drawings illustrating waveforms.

B1. Object

FIG. 44 shows a digital audio input signal 700 of an original sound usedin a game. The audio signal 700 is read from a music CD or the like andstored in the long buffer 283. The audio signal 700 is shown in ananalog waveform for the purpose of brevity. In this example, amplitudevalues are shown in the range form the minimum value −0.5 to the maximumvalue of +0.5 as defined by the vertical axis. The horizontal axis is atime axis for 1.6 seconds. As described above, the audio signal 700includes positive audio events and negative audio events which arerepeated alternately.

In this game, it is necessary to extract distinctive points in music anddisplay road parts (obstacle objects) corresponding to the extracteddistinctive points on the display 18A synchronously with the music.

Therefore, a system for analyzing a waveform of music for identifyingdistinctive points in the music is needed.

As shown in FIG. 45, the audio signal 700 has distinctive pointsindicated by arrows 702. In reproducing the audio signal 700, thesedistinctive points can be emphasized as attacks in the music. Therefore,it is preferable to extract audio events at the respective distinctivepoints indicated by the arrows 702 by a suitable process.

That is, the audio analyzing process according to the present embodimentis intended to analyze music (the waveform of the audio signal 700 shownin FIG. 44) so as to identify attacks in the music (the distinctivepoints indicated by the arrows 702 in FIG. 45). In the game, positionsfor displaying road parts on the display 18A are determined based theidentified distinctive points.

When the audio analyzing process is applied to the game according to thepresent invention, it is necessary to select suitable points from thedistinctive points indicated by the arrows 702 in FIG. 45 and eliminatethe remaining unsuitable points depending on game level settings or thelike.

That is, the purpose of the audio analyzing process according to thepresent embodiment is to extract certain final events based on attacks(distinctive points) in the audio signal (music) recorded in a music CDor the like for utilizing the final events in the game according to thepresent invention.

B2. Brief Explanation of Waveform Processing

As described above, FIG. 44 shows a waveform of an audio signal 700which is read from a music CD or the like and stored in the long buffer283.

FIG. 46 shows a waveform of an emphasized signal 704. The emphasizedsignal 704 is obtained by emphasizing rising parts of the waveform,i.e., by emphasizing attacks in the music. The emphasizing process willbe described later in detail. In FIG. 46, amplitude values are shown inthe positive range from 0 to the maximum value of 1.0 as normalized bythe vertical axis. The horizontal axis is a time axis indicatingrespective sampling points.

FIG. 47 shows a waveform of a signal indicating attack events 706. Thesignal is obtained by converting the emphasized signal 704 with athreshold TH (see FIG. 46) to eliminate unnecessary parts of thewaveform.

FIG. 48 shows a waveform of a signal indicating potential events 708.The signal is obtained by dividing the time axis into a plurality ofblocks (select periods) and extracting a peak in each of the dividedblocks.

It is to be understood that the potential events 708 correspond to thedistinctive points indicated by the arrows 702 in FIG. 45.

FIG. 49 shows a signal indicating final events 710. The final events 710are selected from the potential events 708 based on the game system.

FIG. 50 shows positions of the final events in the music (audio signalof FIG. 44). The final events are extracted from the positions indicatedby arrows 712.

B3. Detailed Explanation of Waveform Processing

B3a. Emphasize Process

The emphasize process is intended to obtain the emphasized signal 704 ofFIG. 46 from the audio signal 700 of FIG. 44.

FIG. 51 shows an enlarged view showing a part of the audio signal 700 inFIG. 44. The audio signal 700 is partially extracted and expanded on thetime axis.

The emphasis process can be performed each time a sampling value isobtained. However, for the purpose of brevity, the emphasis process at apoint of time n1 and the emphasis process at a point of time n2 will bedescribed only.

A short period of time, for example, 23 ms before the time point n1 (orn2) is defined as a short term block Ns of n1 (or n2).

Further, a long period of time, for example, 186 ms before the timepoint n1 (or n2) is defined as a long term block Nl of n1 (or n2).

In FIG. 51, it is appreciated that the fluctuation of the waveform islarge near the time point n1 in comparison with the fluctuation near thetime point n2. That is, the time point n1 (the waveform near the timepoint n1) is considered to be more distinctive than the time point n2(the waveform near the time point n2).

The total sum of values of the audio events (sampling values of thewaveform) near the time point n1, i.e., short term power Ps (n1) islarger than the total sum of values of the audio events near the timepoint n2, i.e., short term power Ps (n2). Therefore, the waveform nearthe time point n1 is considered to be distinctive in comparison with thewaveform near the time point n2.

Next, the method of emphasizing the waveform around the time points n1and n2 will be described. In emphasizing the waveform around the timepoints n1 and n2, the long term blocks Nl are taken into consideration.

The degree of the fluctuation of the present waveform can be effectivelyconsidered by comparing the present waveform with the past waveform.That is, if the fluctuation of the past waveform is small, thefluctuation of the present waveform is considered to be comparativelylarge, i.e., the present waveform is considered to be distinctive.

More specifically, in FIG. 51, the total sum of values of the audioevents in the long term block Nl near the time point n1, i.e., long termpower Pl (n1) is smaller than the total sum of values of the audioevents in the long term block Nl near the time point n2, i.e., long termpower Pl (n2). Therefore, the waveform near the time point n1 isconsidered to be distinctive.

As described above, when the ratio of the short term power Ps to thelong term power Pl is large at a time point, the waveform near the timepoint is considered to be distinctive. In FIG. 51, it is possible toanalyze the degree of the fluctuation at the time point n1 from theratio Ps (n1)/Pl (n1), and analyze the degree of the fluctuation at thetime point n2 from the ratio Ps (n2)/Pl (n2). That is, it is possible toemphasize the audio events in the waveform near the time points n1 andn2 from the ratios. The signal emphasized by the above process isdefined as the emphasized signal.

In the example of FIG. 51, since Ps (n1)/Pl (n1) is much larger than Ps(n2)/Pl (n2), the waveform near the time point n1 is considered to bemuch more distinctive than the waveform near the time point n2.

Next, a quantitative method of calculating the total sum of the valuesof audio events, Ps (n), Pl (n) will be described.

In FIG. 52, powers of audio events at respective time points na and nbare defined.

When a value of the audio event at the time point na is M (na), thepower of the audio event at the time point na corresponds to the areashown by a shaded portion defined by the following expression:

M(na)×M(na)=SQUARE(M(na))>0

Similarly, when a value of the audio event at the time point nb is M(nb), the power of the audio event at the time point nb corresponds tothe area shown by a shaded portion defined by the following expression:

M(nb)×M(nb)=SQUARE(M(nb))>0

A total sum of powers of audio events in a short term block at a timepoint n is defined as the short term power Ps (n).

A method of calculating a short term power Ps (n) in a short term blockat a time point n is described below.

For example, at the time point n1 shown in FIG. 51, the short term powerPs (n1) is expressed by the total sum of powers obtained at respectivesampling points q in the short term block Ns (n1). The short term blockNs (n1) indicates a period of time from the time point n1−Ns to the timepoint n1. That is, the short term power Ps is the sum of the squares ofthe short term block's sampling values (SUM SQUARE (M (q))).

A total sum of powers of audio events in a long term block at a timepoint n is defined as the long term power Pl (n).

A method of calculating a long term power Pl (n) in a long term block ata time point n is described below.

For example, at the time point n1 shown in FIG. 51, the long term powerPs (n1) is expressed by the total sum of powers obtained at respectivesampling points q in the long term block Nl (n1). The long term block Nl(n1) indicates a period of time from the time point n1−Nl to the timepoint n1. That is, the long term power Ps is the sum of the squares ofthe long term block's sampling values (SUM SQUARE (M (q))).

In this manner, a short term power Ps (n) and a long term power Pl (n)at a time point (n) can be calculated.

FIG. 53 is a graph showing short term powers Ps of the audio signal 700in FIG. 44. In the vertical axis, 1e+10 signifies 1×e¹⁰ (e is a base ofnatural logarithm).

FIG. 54 is a graph showing long term powers Pl of the audio signal 700in FIG. 44 in addition to the short term powers Ps in FIG. 53 (scalingof the vertical axis is changed).

FIG. 55 is a graph showing an emphasized signal 704. The emphasizedsignal 704 comprises the ratio (Ps/Pl) of the short term power Ps to thelong term power Pl. FIG. 55 and FIG. 46 are the same graph.

B3b. Event Selection Process

The event selection process is intended to partially eliminate theemphasized signal 704 using a threshold TH. That is, parts (ratiosPs/Pl) of the emphasized signal which do not exceed the threshold valueTH are eliminated. Further, the time axis is divided into a plurality ofselect periods. The length of the select period is related to thescrolling speed in the game. Therefore, the length of the select periodis determined based on game level settings.

FIG. 56 shows an emphasized signal indicating attack events 706. Thesignal is obtained by partially eliminating the emphasized signal 704using the threshold TH. The time axis is divided into twelve selectperiods #1 thorough #12.

Then, peak ratios are detected in the respective select periods #1through #12. The peak ratios are defined as the potential events PE. Thearray of the potential events PE is defined as the potential signal 708.

The positions of the potential events PE constituting the potentialsignal 708 are substantially corresponding to the positions of the audioevents of the audio signal 700 indicated by the arrows 702 in FIG. 45.

B3c. Event Shadowing

The event shadowing process is intended to eliminate unnecessarypotential events PE in the game system and to control the game level.

In the event shadowing process, a shadow period SP is determined as aparameter in setting a game level.

Final events FE needed in the game are selected from potential events PEindicating distinctive points of the music.

Specifically, the event shadowing process comprises the following threesteps (steps 1 through 3).

In step 1, a potential event PE in the present shadow period isselected. Then, it is determined whether another potential event PE isincluded in the present shadow period. That is, in step 1, it isdetermined whether a plurality of potential event PE are included in thesame shadow period of the selected potential event PE or not.

If it is determined that another potential event PE is not included inthe shadow period of the selected potential event PE in step 1, controlpasses to step 2.

In step 2, the selected potential event PE is determined as an effectivefinal event. Then, control passes back to step 1 for selecting the nextpotential event PE on the time axis.

If it is determined that another potential event PE is included in theshadow period of the selected potential event PE in step 1, controlpasses to step 3.

In step 3, a potential event PE having the largest peak value isselected in the present shadow period as a final event FE. If two ormore potential events PE having the same peak ratio are included in thepresent shadow period, the earliest potential event PE on the time axisis selected as a final event. The remaining potential events PE areeliminated. Then, the control passes back to step 1.

The above steps 1 through 3 will be described specifically withreference to FIG. 58 (FIG. 58 and FIG. 57 are the same graph). Firstly,a potential event PE in the first shadow period #1 is selected. Thefirst shadow period #1 includes three select periods #1 through #3. Thatis, there are two potential events PE (a potential event PE in the firstselect period #1 and a potential event PE in the second select period#2) in the first shadow period #1.

In this case, as described above, the potential event PE in the firstselect period #1 is selected and the potential event PE in the secondselect period #2 is eliminated in step 3. That is, the potential eventPE in the first select period #1 is extracted as the effective finalevent FE in the first shadow period. Next, the potential event PE in theselect period #4 is selected as the next final event FE, since thepotential event PE in the second select period #2 has already beeneliminated as described above. In the shadow period #4, there are threepotential events PE (the potential event PE in the select period #4, thepotential event PE in the select period #5, and the potential event PEin the select period #6). Then, the potential event PE in the selectperiod #6 is selected as the final event and the other potential eventsPE in the select periods #4 and #5 are eliminated.

Then, control passes back to step 1. There are three potential events PEin the next shadow period #6 (the potential event PE in the selectperiod #6, the potential event PE in the select period #7, and thepotential event PE in the select period #8). In step 3, the potentialevent PE in select period #6 is selected again as the final event FE andother potential events PE in the select periods #7 and #8 areeliminated. Then, control passes back to step 1.

By repeating the above process, as shown in FIG. 59, three effectivefinal events FE can be extracted from eleven potential events PE of FIG.58.

At the positions of the final events FE, obstacle objects 411 or thelike are generated as road parts.

The type of obstacle object 411 or the like is determined by the processwhich was described with reference to FIG. 19.

As described above, the entertainment system as applied to theembodiment according to the present invention comprises the buffer 283,audio signal analyzing means (the CPU 251), and road part generatingmeans (the CPU 251). The buffer 283 stores an audio signal 700 for acertain period of time. The audio signal 700 includes sampling valuesconstituting continuous events, i.e., positive audio events and negativeaudio events. The audio signal analyzing means reads the audio signal700 from the buffer 283 and analyzes the audio events in the read audiosignal 700 as distinctive points so as to select final events FE. Theroad part generating means generates objects such as road parts 411 orthe like to be displayed on the screen of the display 18A.

According to the present embodiment, the CPU 251 as the audio signalanalyzing means has a first function to generate an emphasized signal704 by calculating a ratio of Ps/Pl, i.e., a ratio of a short term power(Ps) in a predetermined short period before a time point (samplingpoint) to a long term power (Pl) in a predetermined long period of timebefore the time point (sampling point) at each of the time points(sampling points) so as to emphasize sampling values obtained in thesampling points.

Further, the audio analyzing means has a second function to partiallyextract the emphasized signal by comparing the values in the emphasizedsignal 704 and a threshold TH. Specifically, parts of the emphasizedsignal 704 having values smaller than the threshold TH is eliminated andthe remaining parts of the emphasized signal 704 having values equal toor larger than the threshold TH are extracted.

Further, the audio analyzing means has a third function to determinepotential events PE by dividing the emphasized signal 704 into aplurality of select periods having a predetermined period of time andselecting peak values in the respective select periods.

Further, the audio analyzing means has a fourth function to select finalevents FE from the potential events PE. Specifically, shadow periodseach having at least two times longer than the select period areassigned in the overall period of the audio signal such that each shadowperiod starts one of the potential events PE. Then, the largestpotential event PE is selected as the final event FE.

Preferably, a short term power is the sum of the squares of samplingvalues in a short term block and the long term power is the sum of thesquares of sampling values in a long term block.

The road part generating means may generate road parts to be displayedon the display 18A based on combinations of positive and/or negativegradients between respective adjoining peaks of the selected finalevents FE.

What is claimed is:
 1. An entertainment system comprising: anentertainment apparatus for executing various programs; a manualcontroller for inputting a manual control request of a user to saidentertainment apparatus; and a display monitor for displaying an imageoutputted from said entertainment apparatus, wherein said entertainmentapparatus comprises: means for analyzing an audio signal; means forgenerating at least one substantially linear line drawing image; meansfor inserting a non-linear line drawing portion based on an result of ananalysis of said audio signal into said substantially linear linedrawing image to generate a modified substantially linear line drawingimage having said non-linear line drawing portion; and means forgenerating a line drawing image of a character object; wherein said linedrawing image of said character object is displayed on said modifiedsubstantially linear line drawing image having said non-linear linedrawing portion on said display monitor.
 2. An entertainment systemaccording to claim 1, wherein said entertainment apparatus furthercomprises means for moving said line drawing image of said characterobject relative to said modified substantially linear line drawing imagehaving said non-linear line drawing portion.
 3. An entertainment systemaccording to claim 2, wherein said entertainment apparatus furthercomprises means for changing said line drawing image of said characterobject into a line drawing image of a different character objectdepending on how said line drawing image of said character object moveson said modified substantially linear line drawing image having saidnon-linear line drawing portion.
 4. An entertainment system according toclaim 1, wherein said entertainment apparatus further comprises meansfor imparting vibrations to said modified substantially linear linedrawing image having said non-linear line drawing portion and said linedrawing image of said character object.
 5. An entertainment systemaccording to claim 1, wherein each of said line drawing images comprisesa three-dimensional line drawing image.
 6. An entertainment systemaccording to claim 1, wherein said audio signal comprises an audiosignal which is supplied to said entertainment apparatus from arecording medium or which is downloaded to said entertainment apparatusvia communication.
 7. An entertainment apparatus for executing variousprograms, said entertainment apparatus being connectable to a manualcontroller for inputting a manual control request of a user to saidentertainment apparatus, and connectable to a display monitor fordisplaying an image outputted from said entertainment apparatus, whereinsaid entertainment apparatus comprises: means for analyzing an audiosignal; means for generating at least one substantially linear linedrawing image; means for inserting a non-linear line drawing portionbased on a result of an analysis of said audio signal into saidsubstantially linear line drawing image to generate a modifiedsubstantially linear line drawing image having said non-linear linedrawing portion; and means for generating a line drawing image of acharacter object; wherein said line drawing image of said characterobject is displayed on said modified substantially linear line drawingimage having said non-linear line drawing portion on said displaymonitor.
 8. A recording medium for storing a program comprising thesteps of: analyzing an audio signal; generating at least onesubstantially linear line drawing image; inserting a non-linear linedrawing portion based on a result of an analysis of said audio signalinto said substantially linear line drawing image to generate a modifiedsubstantially linear line drawing image having said non-linear linedrawing portion; and generating a line drawing image of a characterobject on said modified substantially linear line drawing image havingsaid non-linear line drawing portion.
 9. A recording medium according toclaim 8, wherein said program further comprises the step of moving saidline drawing image of said character object relative to said modifiedsubstantially linear line drawing image having said non-linear linedrawing portion.
 10. A recording medium according to claim 9, whereinsaid program further comprises the step of changing said line drawingimage of the character object into a line drawing image of a differentcharacter object depending on how said line drawing image of saidcharacter object moves on said modified substantially linear linedrawing image having said non-linear line drawing portion.
 11. Arecording medium according to claim 8, wherein said program furthercomprises the step of imparting vibrations to said modifiedsubstantially linear line drawing image having said non-linear linedrawing image portion and said line drawing image of said characterobject.
 12. A recording medium according to claim 8, wherein each ofsaid line drawing images comprises a three-dimensional line drawingimage.
 13. A recording medium for recording programs and data used foran entertainment system comprising: an entertainment apparatus forexecuting various programs; a manual controller for inputting a manualcontrol request of a user to said entertainment apparatus; and a displaymonitor for displaying an image outputted from said entertainmentapparatus, wherein said program comprises the steps of: analyzing anaudio signal; generating at least one substantially linear line drawingimage; inserting a non-linear line drawing portion based on a result ofan analysis of said audio signal into said substantially linear linedrawing image to generate a modified substantially linear line drawingimage having said non-linear line drawing portion; and generating a linedrawing image of a character object on said modified substantiallylinear line drawing image having said non-linear line drawing portion.14. A program comprising the steps of: analyzing an audio signal;generating a substantially linear line drawing image; inserting anon-linear line drawing portion based on a result of an analysis of saidaudio signal into said substantially linear line drawing image togenerate a modified substantially linear line drawing image having saidnon-linear line drawing portion; and generating a line drawing image ofa character object on said modified substantially linear line drawingimage having said non-linear line drawing portion.
 15. A program for useof an entertainment system comprising: an entertainment apparatus forexecuting various programs; a manual controller for inputting a manualcontrol request of a user to said entertainment apparatus; and a displaymonitor for displaying an image outputted from said entertainmentapparatus; wherein said program comprises the steps of: analyzing anaudio signal; generating at least one substantially linear line drawingimage; inserting a non-linear line drawing portion based on a result ofan analysis of said audio signal into said substantially linear linedrawing image to generate a modified substantially linear line drawingimage having said non-linear line drawing portion; and generating a linedrawing image of a character object on said modified substantiallylinear line drawing image having said non-linear line drawing portion.